Methodology · PreFlight · 14 minute read
A project quality score for SAP delivery: why we built one, and how it works.
Most SAP programmes discover quality problems at UAT. By then the cost of resolution is multiples of what it would have been in build. PreFlight is the instrument set we use to surface those problems earlier, and to score them so a sponsor can read project health at a glance.
Why most SAP projects discover quality problems at UAT
The pattern is consistent across programmes. Build phases close on time. Configuration is reviewed against requirements; integrations are unit-tested; data migrations pass the dry-run. The team moves into User Acceptance Testing, and the dashboard goes red.
What surfaces at UAT is rarely a single defect. It is a cluster: gaps in configuration coverage, integration paths that work in isolation but fail at scale, data quality issues that survived migration, governance debt that accumulated through unrecorded scope changes. The cluster forces a re-plan, a slip, and, sometimes, a partial re-implementation.
None of this is new. Every methodology document warns against it. SAP Activate spends entire pages on entry and exit criteria for each phase. The reason the pattern persists is not that practitioners don't know. It's that the measurement is qualitative and the rhythm is wrong. "Configuration looks complete" is not a measurement. A phase-gate review held quarterly is not a rhythm.
PreFlight is our attempt to fix both. It is a set of four quantitative instruments, a composite scoring model that rolls them into one number, and a threshold rule that ties the score to a recommended action. We use it on every Raptors programme. We also offer it as a diagnostic to customers running programmes someone else delivered.
The four instruments
Each instrument measures one dimension. The four are deliberately orthogonal: a programme can score well on three and badly on one, and the threshold rule will catch it.
Requirements traceability
Configuration coverage
Percent of in-scope configuration objects built, reviewed and tested against requirement. Measured per SAP Activate phase.
The layer that fails at scale
Integration readiness
Percent of in-scope integrations built, end-to-end tested at realistic data volume and exception-tested against known failure modes.
Clean on go-live day
Data quality posture
A composite of migration-ready percent, post-migration validation coverage and known issue residual count.
The audit trail
Governance discipline
Scope-change documentation, decision-log completeness, risk register currency and phase-gate evidence.
The PreFlight score was the first number on the programme that I trusted. Everything else was a story.
The composite score, and why a single number matters
The four instruments roll into one composite, scored 0-100. The composite is weighted: configuration coverage 30%, integration readiness 30%, data quality posture 25%, governance discipline 15%. The weights are deliberate. They reflect the relative cost of each dimension when it goes wrong at UAT.
Exhibit · The composite score
One weighted score rolls four instruments into project health.
Configuration, integration, data and governance combine into a single 0 to 100 reading, recalculated weekly.
82 / 100
composite, recalculated weekly
Score to action
Source: Raptors PreFlight methodology, composite weights and threshold bands as published. Gauge reading illustrative.
The argument for a single number is not that the number is sufficient. It isn't. The instruments matter individually, and a triage conversation reads them individually. The argument is that a single number is sufficient at the steering-committee surface. A programme's executive sponsor does not want a five-dimensional radar chart. They want a current health reading they can act on.
The composite is recalculated weekly. The trajectory matters as much as the level: a programme at 78 trending down is a different conversation from a programme at 78 trending up.
The threshold rule (≥85, 70-84, 50-69, <50)
The score-to-action mapping is the part that makes the measurement operational. Four bands, four recommended actions.
85 and above
Approve the gate
On track against the entry criteria of the next phase. The next phase gate can be approved on current evidence.
Green70 to 84
Monitor
Specific instruments are below target but the composite is defensible. The gate may proceed with a weekly remediation plan as condition.
Amber50 to 69
Intervene
Below the threshold a sponsor can defend in steering. The gate is held until a documented intervention closes the gap.
AmberBelow 50
Re-plan
Materially behind. A re-plan is the only honest path, starting from a fresh PreFlight reading on the new dates.
RedThe rule is harder to game than narrative reporting. A programme cannot talk its way past 50.
What gets measured per Activate phase
PreFlight is built around the SAP Activate phase model. Each phase has its own instrument-level targets and its own composite threshold for the gate.
Discover and Prepare
Governance and data quality posture lead. Configuration targets stay light, but the requirements register has to be measurable.
Explore
Configuration coverage scales into the dominant instrument. Integration readiness starts to register and data quality matures with migration prep.
Realize
All four instruments are at full target weight. This is the phase the composite is optimised for.
Deploy
Configuration and integration readiness hold at target. Data quality shifts to cutover readiness; governance includes the go-live decision pack.
Run
Instruments shift to operational metrics. PreFlight becomes the input to the AMS Wave Health view.
Applying PreFlight to a programme that's not yours
The diagnostic mode is where customers usually first meet PreFlight. A programme is in trouble. Another SI is running it. The customer wants a second reading that isn't from the team in the chair.
4 to 6 weeks
Of programme access
2
Senior practitioners
1
Composite score
4
Instrument readings
We instrument the programme without disrupting delivery. Four to six weeks of access, two senior practitioners, one composite score and four instrument readings at the end. The output is a recommendation (proceed, intervene, re-plan) with the evidence trail.
We've run diagnostic-mode PreFlight on programmes we then took over and on programmes we did not. The honesty of the framework is the point. A diagnostic that always recommends "you should switch to us" is not a diagnostic.
Where to go next
If you're running an SAP programme, or sponsoring one, or about to sponsor a recovery of one, PreFlight is the instrument set we'd use ourselves on day one. Two artefacts start the conversation: your current phase plan and the most recent dashboard report. We can run a two-hour PreFlight read against them and tell you where the signal is.
See the PreFlight service page for the instrument detail and dashboard examples, or talk to our team directly.
Next step