02 · Validate and gate a document offline

You will learn

Read every schema problem at once, each with a pointer
Run the gates in fallback and strict mode
Recognise the hard stops no mode turns off
Check a sealed record with Merkle proofs

You need

Python 3.12 and uv
Notebook 01, or the estimand idea from it

Run this notebook Offline

notebooks/02_validate_and_gate_offline.ipynb

Runs offline on your laptop. The outputs on this page were saved from a real run.

About 15 minutes

Access needed. The packages and notebooks are not yet publicly distributed. The steps below need a checkout of the UMP repository. Pages show recorded outputs, so you can follow along without running them.

cd notebooks && uv run jupyter lab

Runs offline. Uses babelfin (import babelfin as fin).

Priya has a vendor’s summary table for Northwind: ROI per channel, with an interval for TV and a bare number for search. Before she files it with ump-core she wants to know, on her own laptop, whether the contract accepts it and what the gates will say. Core runs exactly these checks on every transition, so what she sees here is what the platform will do.

This notebook covers:

fin.validate: every schema problem at once, each with a pointer to the field;
fin.check: the gates in fallback and strict mode, read as a table;
the hard stops no mode turns off;
fin.Log: why anyone can check a sealed record without trusting the platform.

In [1]:

import babelfin as fin

1. Validate against the contract

Priya’s first attempt, typed from the vendor PDF.

In [2]:

draft = {
    "name": "Northwind vendor ROI table (Acme, FY26)",
    "client_id": "northwind-outdoor",
    "tier": "Z",
    "engine": "vendor:acme",
    "representation": "interval",
    "summaries": [
        {"channel": "tv", "metric": "roas", "value": 1.1, "lo": 0.8, "hi": 1.4, "level": 0.9},
        {"channel": "search", "metric": "roi", "value": "2.0x"},
    ],
}
try:
    fin.validate("model_manifest", draft)
except fin.InvalidDocument as problem:
    print(problem.detail)
    display(problem.to_frame())

Out [2]:

model_manifest doc failed validation (3 issues)

	pointer	message
0	/tier	Input should be 'A', 'B', 'C', 'D' or 'X'
1	/summaries/0/metric	Input should be 'roi' or 'mroi'
2	/summaries/1/value	Input should be a valid number, unable to pars...

Fix the three fields and validation returns the typed contract model, defaults filled in.

In [3]:

doc = {
    **draft,
    "tier": "D",
    "summaries": [
        {"channel": "tv", "metric": "roi", "value": 1.1, "lo": 0.8, "hi": 1.4, "level": 0.9, "current_spend": 6.2e6},
        {"channel": "search", "metric": "roi", "value": 2.0, "current_spend": 4.1e6},
    ],
}
manifest = fin.validate("model_manifest", doc)
print(f"Tier {manifest.tier} {manifest.family} model; acceptance {manifest.acceptance.status}")

Out [3]:

Tier D media_mix model; acceptance untested

2. Run the gates

fin.check runs what core runs when a draft is submitted. In the default fallback mode a gate that finds a problem degrades: it labels the issue, widens an interval or lowers the trust level, and lets the work continue. The method has not been through the acceptance bar, so G-ACC-01 labels it untested; search has no interval, so G-UNC-01 imputes one; nobody validated the vendor table, so G-VEN-01 marks it down. fin.check sees the doc the way core stores it, defaults filled, so these are the same fallbacks the platform will show on submit.

In [4]:

report = fin.check("model_manifest", doc)
report

Out [4]:

Would pass · 0 stops · 3 fallbacks · 0 warnings · 14 gates (4 fired)

next state: checks_passed

details

gate	outcome	message	codes
G-ACC-01	fallback	acceptance untested: usable, labelled unvalidated method, trust reduced	UNVALIDATED_METHOD
G-UNC-01	fallback	missing uncertainty: summary search	UNCERTAINTY_IMPUTED
G-VEN-01	fallback	vendor table unvalidated: trust level 1, imputed band	VENDOR_UNVALIDATED
G-CAP-01	route	sandbox output: experimental label and a named approver needed	EXPERIMENTAL_LABEL

Each fallback says what was assumed and how the numbers moved, so an approver can accept it by name.

In [5]:

report.fallbacks.to_frame()

Out [5]:

	gate	code	field	assumed	effect
0	G-ACC-01	UNVALIDATED_METHOD	NaN	unvalidated method	trust level reduced by one; labelled
1	G-UNC-01	UNCERTAINTY_IMPUTED	summary search	library spread or tier-default band (proposed)	interval imputed, labelled
2	G-VEN-01	VENDOR_UNVALIDATED	NaN	trust level 1	band imputed from the library if none supplied

Strict mode turns every fallback into a stop (the API’s X-UMP-On-Gate-Fail: strict). You can also make only some gates strict.

In [6]:

strict = fin.check("model_manifest", doc, strict=True)
strict

Out [6]:

Stopped · 3 stops · 0 fallbacks · 0 warnings · 14 gates (4 fired)

next state: checks_failed

details

gate	outcome	message	codes
G-ACC-01	strict_stop	acceptance untested: usable, labelled unvalidated method, trust reduced	UNVALIDATED_METHOD
G-UNC-01	strict_stop	missing uncertainty: summary search	UNCERTAINTY_IMPUTED
G-VEN-01	strict_stop	vendor table unvalidated: trust level 1, imputed band	VENDOR_UNVALIDATED
G-CAP-01	route	sandbox output: experimental label and a named approver needed	EXPERIMENTAL_LABEL

In [7]:

try:
    strict.raise_for_stops()
except fin.Stopped as stop:
    print("stopped:", stop.code, "-", stop.detail)

Out [7]:

stopped: UNVALIDATED_METHOD - G-ACC-01: acceptance untested: usable, labelled unvalidated method, trust reduced

In [8]:

fin.check("model_manifest", doc, strict="G-UNC-01").to_frame()

Out [8]:

	gate	outcome	message	codes
0	G-UNC-01	strict_stop	missing uncertainty: summary search	UNCERTAINTY_IMPUTED
1	G-ACC-01	fallback	acceptance untested: usable, labelled unvalida...	UNVALIDATED_METHOD
2	G-VEN-01	fallback	vendor table unvalidated: trust level 1, imput...	VENDOR_UNVALIDATED
3	G-CAP-01	route	sandbox output: experimental label and a named...	EXPERIMENTAL_LABEL

3. Hard stops

Eight gates stop in every mode: privacy (G-PRIV-01..03), record integrity and approver identity (G-INT-01..04) and agent tool separation (G-NEW-02). Priya pasted the vendor contact’s email into the notes, and G-INT-04 refuses to seal a record that holds personal data, strict or not.

In [9]:

leaky = {**doc, "notes": [{"text": "questions to priya.shah@acme-media.example"}]}
fin.check("model_manifest", leaky, at="seal").to_frame()

Out [9]:

	gate	outcome	message	codes
0	G-INT-04	hard_stop	personal data in leaf at ['/notes/0/text']	INTEGRITY_FAILURE

4. The Merkle log

Every validated record becomes a leaf in an append-only Merkle log. Leaves hold ids, hashes and salted references only, never personal data. Anyone with a leaf, its proof and the root can check inclusion.

In [10]:

log = fin.Log()
records = [{"artifact_id": f"art_{i:03d}", "state": "validated"} for i in range(7)]
for r in records:
    log.add(r)
proof = log.proof(4)
proof

Out [10]:

Leaf 4 of 7

audit path of 3 nodes; verifies: True

details

leaf hash	dbed07b27efac21c6270c218bf3c978f771abf360a2c2580ffb9e39f94be0ef9
root	f8c030fe8df15c3688e8b5e912ebe835b36729458a157a6f7a00b1323b2fa5bf

Change one field of record 4 after the fact and the proof no longer fits it.

In [11]:

print("original record:", proof.verify(records[4]))
print("tampered record:", proof.verify({**records[4], "state": "draft"}))

Out [11]:

original record: True
tampered record: False

And a later log can show it only appended to an earlier one (a consistency proof): history was not rewritten.

In [12]:

size, root = log.size, log.root
log.add({"artifact_id": "art_007", "state": "validated"})
print(f"log grew from {size} to {log.size}; only appended: {log.appended_since(size, root)}")

Out [12]:

log grew from 7 to 8; only appended: True

Going deeper

report.results holds every ump_spec gate result, and ump_spec.gates.evaluate is the function underneath.

Next: 03 · From a pandas workflow to a UMP model builds a document like this one from your own model in pandas.