Capability lifecycle and model families
New model types move from sandbox to pilot to certified, with the label on every number changing as they go, and each family declares the checks it must pass; withdrawal is one event that lineage turns into a complete exposure list.¶
Lifecycle¶
| Lifecycle stage | Who sees outputs | Label | Required | Target turnaround |
|---|---|---|---|---|
| Sandbox¶ | Builder and reviewers | Experimental, not for decisions | Spec validates; contract kit passes; registry entry | Same day |
| Pilot¶ | One named account and approver | Pilot method, plus its trust label | Concept review; acceptance layers 1 and 2 on fixtures; null battery on the client’s data | Two weeks |
| Certified¶ | Whole organisation | Normal trust tags | Full acceptance bar (G-ACC-01); independent code review; about card; fixtures in the corpus; owner and refresh schedule | Four to six weeks |
| Restricted¶ | Stated use only | Benchmark only | Kept for comparison, never behind a budget decision | n/a |
| Withdrawn¶ | Nobody, for decisions | Withdrawn, on every past output | Triggered by a failed check, failed decision review or owner | Immediate |
Three axes. An output’s workflow state (section 15), its capability’s lifecycle stage (this section) and its evidence trust (section 0, Terminology) are independent fields. A decision at the Full materiality tier MAY rest only on outputs that are validated and come from a Certified capability, or from a Pilot capability for its named account only, with trust not excluded.¶
Withdrawal is an overlay. capability.withdrawn writes a new record and lineage flags. Consumers MUST apply the withdrawn label when they read a past output, and MUST NOT rewrite the stored output or its log leaf.¶
Deprecation. A capability or plugin version is deprecated by a capability.deprecated event naming a removal date at least 90 days later (proposed). Until that date it stays usable, and consumers MUST surface a deprecation note. After it, the version is treated as Restricted. Deprecation is planned retirement. Withdrawal is removal for cause and takes effect immediately.¶
Gates. G-CAP-01 sandbox output reaches a client only with an experimental label and a named approver; withdrawn output is excluded and the step falls back to the next capability. G-CAP-02 pilot outputs carry the label and the pilot account’s approver. G-CAP-03: every new capability version MUST rerun its known-truth and null fixtures in CI. G-CAP-04 a registry entry matching an existing question needs a written reason to fork. G-CAP-05 agents run capabilities only where status allows.¶
Containment is tested. On capability.withdrawn, lineage flags every report, scenario and decision that used the capability and lists every exposed account. The conformance suite injects a withdrawal into a fixture graph and fails if any downstream artifact is missed.¶
Duplication. Every capability gets one registry entry written from its manifest; a new proposal starts with a search. Discovery is not enough on its own, so certification is designed to be cheaper than bypass: acceptance pipelines and evidence packs are pre-built per family, and cycle time is a tracked metric. Capabilities ship as hash-pinned modeling-api expert plugins with semver ranges and a deprecation window.¶
Family profiles. A profile is data: the shared names it needs, extra requests, how it is calibrated, and its acceptance checks.¶
| Family | Extra requests | Calibrated by | Family acceptance checks |
|---|---|---|---|
| Media mix¶ | response_curve, decompose, optimize | Admissible geo and user tests via section 10 | Full bar, section 14 |
| Audience¶ | rank, size, overlap, uplift | User-level holdouts, ghost ads, ITT with CACE | Model declares propensity or uplift; uplift validated only on randomised holdouts, never Qini or AUUC on observational data; known-truth simulation; minimum segment size |
| Reach and frequency¶ | reach_curve, frequency, dedupe | Panel reach studies | Coverage against panel truth |
| Attribution¶ | attribute, path_summary | Lift tests; triangulation with MMM | Agreement with experiments; shuffled-path nulls |
| Brand and funnel¶ | track, decompose_latent | Survey brand lift (class E) | The MTM bar: recovery, nulls, real segment and period differences |
| Forecasting¶ | forecast, backtest | Held-out periods | Beats naive; interval coverage |
| Creative¶ | score_creative, wearout | Creative tests with divergent-delivery controls | Content effect separated from delivery |
Audience and MMM coherence (research item). Segment-level uplift summed over segments should reconcile with the channel’s MMM contribution within stated uncertainty. No standard method exists; until one is validated, the two are shown side by side with any gap flagged, and neither overrides the other.¶