Gap closure register
This section is informative.¶
All ten ranked gaps now have a normative rule and a named control (a gate, a computed tier that sets review depth, or a logged sign-off); eight still need evidence before the rule’s numbers are final, and none is marked closed until that evidence is filed.¶
| # | Gap (review rank) | Rule now in the spec | Control | Evidence still needed | Residual risk |
|---|---|---|---|---|---|
| 1¶ | Estimand identity not in the contract | Seven-field estimand on every output; mismatches aligned through a converter, a re-asked model or a widened assumption, never silently (s.5) | G-EST-01, G-EST-02, ESTIMAND_ALIGNED | Study 1 confirms the framework map | Adapters may misread a framework’s defaults |
| 2¶ | Lower tiers lose joint dependence | Dependence declaration; without joint draws, portfolio maths runs on an independence-to-comonotone envelope (s.7, 8) | G-DEP-01, DEPENDENCE_ASSUMED | None for the rule; study 1 tests labelled assumptions | Declared copulas may be wrong; they are labelled |
| 3¶ | Acceptance bar tests computation, not identification | Three layers including endogenous-spend simulations; decision-term pass marks; pre-registration; Holm control (s.14) | G-ACC-01 (per-family battery, section 14) | Study 2 calibrates pass marks | Simulators cannot cover every real confounder |
| 4¶ | Vendor curves unvalidated | Outcomes analysis against admissible tests; attach-only calibration (s.10, 14) | G-VEN-01, VENDOR_UNVALIDATED | Enough admissible tests per account | Accounts with few tests stay on unvalidated tables |
| 5¶ | Calibration mechanics | PSIS rule, excision with adstock tail, granularity mapping, BH conflicts, interval lift credited and at pilot (s.10) | G-CAL-01 to 05, PSIS_UNRELIABLE | Study 3 | Joint-model alternative to excision is untested |
| 6¶ | No evaluation of Newton | Harness as release gate; capability separation; pre-registered methods; labelled best guesses instead of silence (s.17) | G-NEW-01 to 03 | Study 5, then every release | New injection techniques outpace the red team |
| 7¶ | Portfolio maths overclaimed | Cost-scaled greedy on EIG with best-single fallback; joint EVOI check; no bound claimed for EVOI (s.11) | Logged measurement-lead sign-off on priority_snapshot | None | Joint EVOI is costly at enterprise scale |
| 8¶ | Platform tests not classed | Admissibility classes A to E; class D excluded from incrementality calibration while the step continues (s.9) | G-EXP-05, EVIDENCE_EXCLUDED | Study 6 informs class C discounts | Platform Lift methods change without notice |
| 9¶ | Linear hash chain | Merkle log, RFC 3161 and external anchors, independent verifier, key versions, off-log personal data (s.18) | G-INT-01 to 04 | Study 8 | Collusion across three separated roles |
| 10¶ | Superseded guidance, no materiality tiers | SR 26-2 and NIST AI RMF anchoring; tiers by spend, reversibility and EVOI (s.15) | G-MAT-01 (computed tier that sets G-MOD-01 review depth) | Study 9; legal review of SR 26-2 scope and EU AI Act | Thresholds set per account may drift |
Other review items addressed. Close-rerun tolerance and environment capture (s.13); extraction error rates and double-keying (s.6); PROV plus OpenLineage (s.6); RFC 9457 (s.19); response-curve exchange format (s.7); capability handshake (s.8); spillover and estimator choice (s.9); Fieller intervals and net benefit (s.9); optimiser’s curse (s.12); propensity versus uplift and differencing protection (s.16, 20); CloudEvents, Standard Webhooks, idempotency and ordering (s.19); tested containment and duplication incentives (s.16); governance health metrics (s.15).¶
Not yet addressed. Coherence between audience uplift and MMM contributions has no method (research item, s.16). Batch and amortised experimental design remain research items (s.11).¶