Governance as code (closes gap 10)
Policies run at every state transition of every artifact kind, review depth is set by the money at stake and how hard the decision is to reverse, and the framework cites SR 26-2 and the NIST AI RMF as its external anchors.¶
State machine. One pure function governs every artifact kind. Its states are draft, checks_running, checks_passed, checks_failed, awaiting_approval, validated, rejected and deprecated. The allowed transitions are: draft to checks_running; checks_running to checks_passed or checks_failed; checks_passed to awaiting_approval; awaiting_approval to validated or rejected; validated to deprecated; and checks_failed or rejected back to draft, which starts a new version. Any other transition MUST be refused with INVALID_TRANSITION. Transitions into validated or rejected need an approval. Every transition is audited and emitted as an event. Reviewers cannot approve their own submissions; the submitter, the artifact’s owner and its listed authors all count as self (G-INT-02, SELF_APPROVAL_FORBIDDEN). No API token can approve (Newton’s included), and a rejection needs a comment. An approver who signs while review requirements of the materiality tier are open MUST give an override reason, recorded as an override; without one, the approval is refused with REVIEW_REQUIREMENTS_OPEN, a validation error and not a gate. The design follows modeling-api’s workflow, which is acknowledged here; this section is the normative definition.¶
-
Draft Work in progress.
-
Checks running The policies run. Every move between states is audited and sent out as an event.
-
Checks passed Ready to ask for approval.or Checks failed
-
Awaiting approval Waits for a named person to sign.
-
Validated Approved by a named person. Agent read tools see only validated artifacts.or Rejected Needs a comment.
-
Deprecated Retired from use.
One state machine for every kind of artifact. A model, a test result, a scenario and a decision all follow the same path, so a reader can always tell how far a number has come.
Three rules decide who can move it on:
- Reviewers cannot approve their own submissions.
- No API token can approve, Newton's included.
- A rejection needs a comment.
Approvals are signed by the named approver, never the submitter or a token G-INT-02. Breaking that rule is a hard stop.
Policies. modeling-api’s shape: a tenant floor projects can only tighten; outcomes hard stop, fall back, route, warn and info (defined below); a version recorded with every validation. UMP adds applies_to.kinds and route_to. Policies are written in a declarative form compiled to Open Policy Agent Rego, and the Rego files with their unit tests ship in the conformance suite.¶
Materiality tiers. Marketing has no supervisor, so tiers must be self-defined and defensible. Each decision record and the models behind it get a tier from three inputs (G-MAT-01): spend at stake, weeks to reverse, and the expected value of perfect information on the decision, computed as in section 11. Where EVOI is a range, the upper end is used, so missing dependence buys more review, not less. The cost of a full review is an account parameter (proposed, set per account).¶
| Tier | Assigned when (defaults proposed, set per account) | Review |
|---|---|---|
| Light¶ | Spend at stake below the standard threshold and reversible within a month | Owner checks against last version; reviewer reads the decision-log diff |
| Standard¶ | Spend at stake at or above the standard threshold, or feeds a client recommendation | Reviewer signs off plan before fit and outputs before release |
| Full¶ | Spend at stake at or above the full threshold, irreversible within a quarter, a public claim, or EVOI on the decision above ten times the cost of a full review | Standard plus an independent rerun from raw data |
The tier is a deterministic function, evaluated in order. It is Full if any Full trigger holds. Otherwise it is Standard if spend at stake is at or above the standard threshold, the decision feeds a client recommendation, or reversal takes more than 4 weeks (proposed). Otherwise it is Light. “Irreversible within a quarter” means more than 13 weeks to reverse (proposed). Weeks to reverse are recorded on the decision record.¶
- Light When: spend at stake below the standard threshold and reversible within a month. Review: the owner checks against the last version; a reviewer reads the decision-log diff.
- Standard When: spend at stake at or above the standard threshold, or it feeds a client recommendation. Review: a reviewer signs off the plan before the fit and the outputs before release.
- Full When: spend at stake at or above the full threshold, irreversible within a quarter, a public claim, or EVOI on the decision above ten times the cost of a full review. Review: Standard, plus an independent rerun from raw data.
The tier comes from three inputs G-MAT-01: spend at stake, weeks to reverse, and the expected value of perfect information (EVOI) on the decision. Each decision record and the models behind it get one.
Deeper review is bought where the expected loss from a wrong call is larger than its cost. Threshold defaults are proposed and set per account.
The EVOI clause makes the rule defensible: deeper review is bought where the expected loss from a wrong call is larger than its cost.¶
Regulatory anchoring. On 17 April 2026 the Federal Reserve, OCC and FDIC issued SR 26-2, which supersedes SR 11-7 and SR 21-8. UMP borrows its principles (effective challenge, conceptual soundness, ongoing monitoring, outcomes analysis, equal treatment of vendor models) without being subject to it. Newton is governed under the NIST AI RMF 1.0 and the Generative AI Profile (NIST AI 600-1). Most mix-model work is unlikely to be high-risk under the EU AI Act, but transparency and deployer duties may apply; legal review is open.¶
Note (informative, pending legal review, section 24): the scope of SR 26-2 for generative and agentic AI has not been confirmed against the primary text.¶
NIST crosswalk (informative). Each agent control maps to an AI RMF function and to the AI 600-1 risk it addresses.¶
| Control | AI RMF function | AI 600-1 risk |
|---|---|---|
G-NEW-01 agent cites an artifact for every number¶ | Measure | Confabulation; information integrity |
G-NEW-02 sessions that read untrusted text hold no write or run tools¶ | Manage | Information security |
G-NEW-03 agent version passes the harness before release¶ | Govern | Confabulation |
G-CAP-05 agents run capabilities only where status allows¶ | Manage | Information security |
| Agent evaluation harness (section 17)¶ | Measure | Confabulation; information integrity |
| Injection red team (section 17)¶ | Measure | Information security |
Subcategory identifiers are added after verification against NIST AI 100-1.¶
Outcomes analysis, monitored. Realised versus predicted lift for every new admissible test; realised versus expected uplift at each decision review; vendor table agreement (section 14). These are standing metrics on the governance dashboard.¶
How a gate fails: fall back first. A gate that stops work does not make the decision go away. The planner still has to set a budget on Thursday, so a blocked step usually ends in a spreadsheet with no label at all, which is the failure UMP exists to prevent. Gates therefore default to the best answer the available evidence supports, and say exactly what was assumed.¶
- Hard stopThe step does not run. Only eight gates: privacy, record integrity and approver identity, agent tool separation.
- Exclude and continueEvidence known to be wrong is left out; the step carries on with what remains.
- Fall backRuns on the next-best evidence or a registered default, with a
fallbacksentry and a wider interval. Most gates. - RouteRuns now as provisional; a named reviewer confirms, revises or reverts.
- WarnRuns with a visible note attached.
Gates degrade before they block. A blocked step does not make the decision go away: the plan still has to be set on Thursday. So a failed gate gives the best answer the evidence supports and says what it assumed.
A request can set on_gate_fail: "strict" for named gates, which turns their fallback into a stop with a typed error.
| Outcome | What happens | Used for |
|---|---|---|
| Hard stop¶ | The step does not run. | Eight gates where going ahead is unsafe, not merely less certain: privacy (G-PRIV-01 to 03), record integrity and approver identity (G-INT-01 to 04), and agent tool separation (G-NEW-02) |
| Exclude and continue¶ | Evidence known to be wrong is left out, and the step carries on with what remains, one rung down the fallback ladder. | A capability that failed the acceptance bar, a contaminated test, a platform A/B test offered as incrementality evidence |
| Fall back¶ | The step runs on the next-best evidence or a registered default assumption. The result carries a fallbacks entry and, wherever the effect can be quantified, a wider interval. | Most gates |
| Route¶ | The step runs now and its output is usable as provisional; a named reviewer confirms, revises or reverts it. | Judgment calls: conflicts, post-hoc choices, unreviewed plans |
| Warn¶ | The step runs; a visible note is attached. | Design and context issues |
Hard stops fail closed. Each hard-stop gate binds to a fixed set of operations: G-INT-01 to every write of a sealed record; G-INT-02 to every transition into validated or rejected; G-INT-03 to every export and every agent answer that cites a sealed record; G-INT-04 to every leaf append; G-PRIV-01 to 03 to every aggregate or score leaving the query layer; and G-NEW-02 to every agent tool call. For a bound operation, an input the gate needs that is missing, malformed or unverifiable MUST evaluate as a failure, never as not applicable. Not applicable is allowed only for operations outside the binding. An export carrying G-REP-01’s untraced marker MUST NOT claim a sealed citation or validated status. Hard stops MUST NOT be lifted by strict-mode settings, policy floors, overrides or reviewer action; a request to override one is refused with HARD_STOP_NOT_OVERRIDABLE (section 19).¶
The distinction between falling back and excluding matters. Weaker evidence about the right question is still evidence, so it is used with a wider range. A model that ranked drivers no better than chance, like MTM, is not weak evidence; it is noise with a confident label, so it is excluded and the step falls back to the next source.¶
Every fallback leaves a trace. Each result lists its fallbacks (gate, what was assumed, the effect on the interval). Decision records show them to the approver, who acknowledges each one for decisions at the Standard and Full materiality tiers; acknowledgement is required, but it never blocks. Briefs and agent answers name them in plain words, for example “assumes the vendor table measures return against zero spend”.¶
Widening, not just labelling. Where an assumption can be bounded, the interval covers the bound: the envelope across plausible values of an assumed estimand field, the span between independent and perfectly correlated channels for an assumed dependence, the tool’s measured error rate for unchecked extraction. A label alone tells the reader to worry; a wider range tells them how much. The widening operator is defined in section 5 (Envelope).¶
Strict mode. Rigor is sometimes the point, for a public claim or a client contract that demands it. A request or an account policy may set on_gate_fail: "strict" for named gates, which turns their fallback into a stop with the usual typed error. The default is "fallback". on_gate_fail is either the string "fallback" or "strict", applying to all gates, or an object mapping gate IDs to either value, with "*" as the default key. Strict affects only gates whose outcome is Fall back. Exclude and continue, Route, Warn and Computed outcomes are unchanged. A gate whose registry entry has strict_override: false keeps its fallback even in strict mode. The effective mode is the stricter of the request and the account policy: a request MUST NOT relax a gate the account sets to strict. Section 19 defines the carriers.¶
Keeping fallbacks from becoming the norm. The share of decisions resting on each fallback is a governance health metric. A channel that keeps needing the same fallback gains test priority automatically, because the assumption it rests on carries value of information that a test or a better vendor export would buy.¶
Gate catalogue (spec 2.3.0-draft). Eight hard stops; everything else falls back, routes or warns. This catalogue and the benchmark gates in 25.4 together form the gate registry (section 21). Every gate ID in this specification resolves to exactly one row; a gate engine MUST refuse to load a policy whose gate ID resolves to none. The Code column names the note or error code each gate raises.¶
| ID | Rule | If it fails | What happens instead | Code |
|---|---|---|---|---|
| G-DATA-01¶ | Raw delivery registered with fingerprint, source, range | Fall back | The file is registered as received with source marked unregistered; audit level 2 | DATA_UNREGISTERED |
| G-DATA-02¶ | Extracted totals tie to printed totals | Fall back | Extracted values used; the gap is shown (“totals off by 1.8%”) and, above tolerance, added to the interval | TOTALS_GAP |
| G-DATA-03¶ | OCR, AI and hand values above materiality double-keyed | Fall back | Single-checked values used, labelled unchecked, interval widened by the tool’s measured error rate | UNCHECKED_EXTRACTION |
| G-DATA-04¶ | Restated history or new or missing channels vs last delivery | Warn | Latest delivery used; the diff is attached | DATA_RESTATED |
| G-MOD-01¶ | Model plan reviewed before first fit (Standard and Full materiality tiers) | Route | Fit runs; outputs labelled plan not reviewed until the reviewer signs | PLAN_NOT_REVIEWED |
| G-MOD-02¶ | Post-hoc decisions carry a reason; headline movers go to a reviewer | Route | Change applies provisionally; reviewer confirms or reverts | POST_HOC_PROVISIONAL |
| G-MOD-03¶ | No leakage risk in the spec | Fall back | Fit runs; holdout metrics marked unreliable and not counted as acceptance evidence | LEAKAGE_RISK |
| G-ACC-01¶ | Family acceptance bar passed on a pre-registered battery | Fall back or exclude | Not yet run: outputs usable, labelled unvalidated method, trust reduced. Failed: excluded; the step uses the next-best capability or library benchmarks | UNVALIDATED_METHOD, ACCEPTANCE_UNSUBSTANTIATED |
| G-EST-01¶ | All seven estimand fields present | Fall back | Missing fields take the registered default for that source type, marked assumed; interval widened to the envelope across plausible values | ESTIMAND_UNDECLARED |
| G-EST-02¶ | Combined quantities share estimand and scope | Fall back | In order: a registered converter; re-asking the model the other number’s question; combination under an assumed alignment with envelope widening; showing both side by side. Never combined silently | ESTIMAND_ALIGNED |
| G-UNC-01¶ | Uncertainty present; summaries and aggregations allowed by the representation | Fall back | Missing uncertainty imputed from the test library’s spread for that channel family, or a default by tier; summed quantiles reported as bounds; with no registered default, the result is shown without portfolio maths | UNCERTAINTY_IMPUTED |
| G-DEP-01¶ | Portfolio operations on joint dependence | Fall back | Independence assumed; the interval is set to the envelope between independent and perfectly correlated channels, labelled envelope_kind: bound for sums, means, CVaR and monotone functions of sums, and envelope_kind: range for quantile-type, non-monotone outputs and EVOI (section 7) | DEPENDENCE_ASSUMED |
| G-EXP-01¶ | Primary metric, method and decision rule locked before launch | Fall back | Result kept, labelled not pre-registered, given a power-prior discount in calibration | NOT_PREREGISTERED |
| G-EXP-02¶ | Assurance at least 80%; t-quantile MDE; placebo calibration | Warn | Test may launch; the plan shows its chance of an inconclusive result for the approver to acknowledge | LOW_ASSURANCE |
| G-EXP-03¶ | Spillover design recorded; buffer shift tested | Warn | Note attached; contamination flag if the buffer shifts | SPILLOVER_UNRECORDED |
| G-EXP-04¶ | Duration clears the carryover floor | Fall back | Lift adjusted for uncaptured carryover through the adstock kernel per draw, labelled | CARRYOVER_ADJUSTED |
| G-EXP-05¶ | Only classes A to C calibrate sales-return incrementality; class E calibrates the brand family only | Exclude and continue | Class D stays out of calibration (it measures a different thing); the step uses remaining tests or benchmarks | EVIDENCE_EXCLUDED |
| G-CAL-01¶ | Conflict check with multiplicity control | Route | Calibration runs provisionally under a robust mixture prior so one surprising test cannot over-steer; review confirms or reverts | CALIBRATION_CONFLICT |
| G-CAL-02¶ | Excision window including adstock tail recorded | Fall back | Default excision applied automatically and labelled | DEFAULT_EXCISION |
| G-CAL-03¶ | Granularity mapping recorded | Fall back | Default split rule applied and labelled; routed if it moves a headline number | DEFAULT_SPLIT |
| G-CAL-04¶ | Tier and measurement mapping allow the requested route (section 10 check 2a) | Fall back | Attach only: the model is unchanged and the result shown beside it | ATTACH_ONLY |
| G-CAL-05¶ | PSIS k̂ and ESS pass for reweighting | Fall back | Summary-level precision-weighted update (section 10) moment-matched onto the draws, labelled approximate; a refit is queued | PSIS_UNRELIABLE |
| G-VEN-01¶ | Vendor table validated by outcomes analysis | Fall back | Usable at every materiality tier with trust unvalidated; band imputed from the library if none supplied; the approver of a decision at the Full materiality tier acknowledges | VENDOR_UNVALIDATED |
| G-MAT-01¶ | Materiality tier from spend, reversibility and EVOI | Computed | MATERIALITY_COMPUTED | |
| G-SCN-01¶ | Scenario calls a validated endpoint revision | Fall back | Scores on the last validated revision; on request, on the newer revision with a label | NOT_VALIDATED |
| G-SCN-02¶ | Fallback steps 4 and 5 carry a visible warning | Warn | LADDER_SUMMARY_OR_BENCHMARK | |
| G-SCN-03¶ | Spend outside the seen range or grid marked as extrapolation | Warn | Interval doubled (proposed) beyond seen spend | OUTSIDE_GRID |
| G-SCN-04¶ | Deltas use common random numbers and the thinning index | Fall back | Independent draws; delta interval wider, labelled | INDEPENDENT_DRAWS |
| G-REP-01¶ | Every exported number resolves to an artifact ID | Fall back | Exported with an untraced marker and logged as a manual note | UNTRACED |
| G-NEW-01¶ | Agent cites an artifact for every number | Fall back | Cites, or gives a best guess labelled as such with its basis; an uncited number is never presented as cited | UNCITED_BEST_GUESS |
| G-NEW-02¶ | Sessions that read untrusted text hold no write or run tools | Hard stop | TOOL_SEPARATION | |
| G-NEW-03¶ | Agent version passes the harness before release | Fall back | The last passing version stays live; with none, questions go to analysts | AGENT_VERSION_HELD |
| G-PRIV-01¶ | Audience outputs below minimum segment size suppressed | Hard stop | PRIVACY_FLOOR | |
| G-PRIV-02¶ | Member-level scores never leave the clean room or hashed ID space | Hard stop | PRIVACY_FLOOR | |
| G-PRIV-03¶ | Query budget and overlap checks stop differencing | Hard stop | PRIVACY_FLOOR | |
| G-CAP-01¶ | Capability status allows the use | Route or exclude | Sandbox output reaches a client only with an experimental label and a named approver; withdrawn output is excluded | EXPERIMENTAL_LABEL |
| G-CAP-02¶ | Pilot outputs carry the pilot label and the pilot account’s approver (section 16) | Route | Output usable as provisional; the pilot account’s approver confirms | PILOT_LABEL |
| G-CAP-03¶ | Every new version reruns its known-truth and null fixtures in CI (section 16) | Fall back | The previous version stays live | FIXTURES_NOT_RERUN |
| G-CAP-04¶ | A registry entry matching an existing question carries a written fork reason (section 16) | Route | Entry registered provisionally; a reviewer confirms the reason | FORK_NEEDS_REASON |
| G-CAP-05¶ | Agents run capabilities only where status allows (section 16) | Fall back | The agent recommends the run and an analyst runs it | AGENT_RUN_NOT_ALLOWED |
| G-INT-01¶ | Sealed records written only as Merkle log leaves | Hard stop | INTEGRITY_FAILURE | |
| G-INT-02¶ | Approvals signed by the named approver, never the submitter or a token | Hard stop | APPROVER_INVALID, SELF_APPROVAL_FORBIDDEN | |
| G-INT-03¶ | Exports cite records with valid inclusion proofs | Hard stop | INTEGRITY_FAILURE | |
| G-INT-04¶ | Personal data never written into log leaves | Hard stop | INTEGRITY_FAILURE | |
| G-AUD-01¶ | Audit level is the lowest of all inputs | Computed | AUDIT_LEVEL_COMPUTED |
Roles. Model owner, reviewer (never the builder), data owner, approver, framework owner, measurement lead (sole approver of calibrations), experimentation analyst, agent host steward, key custodian (new, section 18).¶
Governance health. Review cycle time, bypass rate (numbers in client work with no artifact ID) and override frequency are tracked for one quarter on the pilot before scaling. If certification is slower than the side road, analysts will take the side road, which is how MTM reached clients.¶