Skip to content
Universal Model ProtocolDocs

Governing the standard

§21NormativeDraft 2.3Latest draft

UMP is published as a versioned specification with JSON Schemas, a contract package, reference code and a conformance suite; a tool conforms when it passes the suite for the classes it claims, and the suite now tests meaning as well as shape.

Parts. Written spec; JSON Schema for every artifact kind; two normative OpenAPI 3.1 documents: the core API (section 19) and the engine profile (section 8). Schemas use JSON Schema 2020-12, with $id values of the form <base>/v<schema major>/<name>.json. $id values are identifiers: a conforming validator MUST NOT require network retrieval, and publishers SHOULD serve each schema at its $id or at a documented mirror. The artifact kinds are a registry, and kind and applies_to.kinds MUST use its values. The publishing base is open (section 24). Other parts: Arrow or Parquet layouts for tables; InferenceData and zarr layout for draws; the ump_spec contract package (schemas, representation rules, error codes, estimand registry) restated or lifted only from Apache-2.0 sources; reference Python service and library, TypeScript client and R reader.

Versioning. Each axis has its own Semantic Versioning 2.0 version and its own field.

  1. The specification: this document is 2.3.0-draft. Every artifact MUST carry it in spec_version. While the specification is a draft, the value carries a pre-release tag, and implementations MUST NOT assume compatibility across pre-release values.
  2. Schemas: the $id path major (currently /v1/) changes only on a breaking schema change.
  3. Each registry (estimands, codes, gates, event types, domain tags, threshold set, benchmark parameters) carries its own version, recorded in the field that cites it (for example the estimand’s schema_version, <registry>/<MAJOR.MINOR.PATCH>).
  4. Gate policies carry a policy version recorded with each validation (section 15).
  5. The conformance suite (below).

Compatibility. A change is MAJOR if it removes or renames a member, narrows a type or a closed enum, adds a required member, changes a default, changes what a value means, or changes a gate’s outcome class. Adding an optional member, or a value to an open enum, is MINOR. Corrections that change neither validation nor meaning are PATCH. Enums are closed unless the schema marks them open. A consumer that meets an unknown value of an open enum MUST treat it as undeclared.

Unknown members. In an artifact whose spec_version MINOR is higher than the reader’s, the reader MUST ignore unprefixed members it does not recognise and MUST preserve them on re-export. A re-export that preserves such members MUST keep the source artifact’s spec_version. At the reader’s own or a lower MINOR, such a member is SCHEMA_INVALID. Inside the estimand and the other value objects of sections 4 and 5, an unknown member is always rejected (ESTIMAND_UNKNOWN_FIELD).

Negotiation. Metadata (section 8) and GET /v1/meta list supported_spec_versions as MAJOR.MINOR ranges. Requests state the version in the media type parameter profile (for example application/json; profile="<base>/spec/2.3"). A reader MUST accept any artifact with the same MAJOR and a MINOR no greater than its own, and MAY read a higher MINOR under the unknown-member rule. While spec_version carries a pre-release tag, a reader MUST accept only the exact pre-release values it lists in supported_spec_versions. The MINOR rule applies from the first release without a tag. It MUST reject an unsupported MAJOR with SPEC_VERSION_UNSUPPORTED (415). When no supported range overlaps the Accept profile, the server returns SPEC_VERSION_NOT_ACCEPTABLE (406), listing its ranges. Upgrade steps are not defined yet; the first MAJOR change MUST publish them.

Conformance suite. Each fixture is a folder holding a data generator, a spec document, a list of requests each with expect or expect_error, and expectations. The layout follows modeling-api’s fixtures, which are acknowledged here. The suite has these fixture families:

FamilyWhat it proves
Schema and round tripArtifacts validate and survive write, read and re-export unchanged; identical content hashes across the Python, TypeScript and R implementations
Estimand round tripInterop study (study 1, run by the maintainers, not a class requirement), as described in section 22. Adapter fixtures: frozen posterior draws or curve parameters supplied as data, with the expected exchanged estimand
Expected fallbacks and stopsQuantile sums, portfolio maths without joint draws, class D calibration and undeclared estimands each return the right fallback note and interval widening, the right typed error in strict mode, and every hard stop stops. For each hard-stop gate, one fixture per required input omits that input and expects the stop
AdmissibilityOne fixture per experiment source type asserts the admissibility class it is assigned (section 9)
MultiplicityBenjamini–Hochberg conflict flags (section 10): m = 1 with z = 1.8 is not flagged and z = 2.1 is flagged; m = 3 with z = (1.8, 1.9, 0.2) gives no flags; m = 3 with z = (2.5, 2.2, 0.2) flags the first two
Acceptance barPositive control passes; MTM negative control fails
Lineage and integrityInjected withdrawal finds all downstream artifacts; tampered leaf and forged head are detected; byte-exact canonicalisation and hash vectors (records, canonical bytes, leaf hashes, roots for tree sizes 1 to 8, inclusion and consistency proofs; including a non-BMP member name, the numbers 1e20 and 1e-7, and an instant), a tampered leaf and a forged checkpoint that a verifier MUST reject, reproduced byte for byte in the Python, TypeScript and R implementations; an erased record verifies as redacted while an edited mirror body fails against content_hash; created, stale and withdrawn events for one revision, delivered out of order and duplicated, leave the consumer showing stale and withdrawn

Policies in Rego and their unit tests are part of the suite.

Suite artifact. The suite is published as its own versioned artifact. Its Semantic Versioning major and minor equal the spec version it tests (suite 2.3.x tests spec 2.3). Fixtures are language-neutral files holding inputs, the request, and either an expected output with an absolute and relative tolerance per asserted field, or an expect_error code. Draw-based outputs use the section 13 close-rerun statistic. Optimiser fixtures compare the objective value within tolerance, not the allocation vector. An implementation under test exposes either the section 19 HTTP API or a CLI that reads a fixture on stdin and writes the response on stdout. A class passes when every fixture tagged with that class, and with a request or tier it claims, passes. A fixture cannot pass partially. The runner writes a JSON report listing each fixture ID and result. A patch release of the suite MAY add fixtures only for behaviour the spec already required. An Engine adapter never needs to run another vendor’s engine. The reference package’s unit tests are not the suite.

Extensions. An artifact body MAY carry vendor members matching ^x-[a-z0-9]+(-[a-z0-9]+)*$, where the first segment is a prefix from the extension-prefix registry. An event MAY carry vendor extension attributes named x + registered prefix + name in [a-z0-9] only (for example xacmeregion), as CloudEvents 1.0.2 requires. Value objects (estimand, interval, unit and the other objects of sections 4 and 5) MUST NOT carry extensions, so that G-EST-02 never depends on unknown fields. Consumers MUST ignore extension members they do not understand, MUST preserve them on round trip, and MUST NOT let them change a gate outcome. An extension that moves into the core gets an unprefixed name, and the prefixed name stays readable for one MAJOR version.

Registries. UMP keeps these registries as normative machine-readable files published with the spec: gate IDs, note and error codes, CloudEvents types, leaf domain tags, artifact kinds, family profiles, estimand converters, extension prefixes, the threshold set and the benchmark parameters. Each entry carries an ID, a status (provisional, active or deprecated) and a spec reference, and, for gates and codes, an outcome class. Gate IDs and codes are Specification Required. Extension prefixes are First Come First Served. Family profiles and converters are Expert Review (RFC 8126 terms). Adding a hard-stop gate, or moving a gate into the hard-stop class, MUST be a spec change, because section 15 fixes the hard stops at eight gates. Entries used by the reference implementation but not yet in this document (for example G-DATA-05 to 07 and G-EXP-06) are provisional. Implementations and publishers MUST NOT present them as normative. The registration authority follows the publishing home (section 24).

Change process. Proposed changes are published for review before adoption. Name, licence and publishing home are open (section 24).