Skip to content
Universal Model ProtocolDocs

Scope, non-goals and conformance classes

§2NormativeDraft 2.3Latest draft

Version 1 covers marketing mix models (Bayesian and frequentist), third-party response curves, geo and user-level experiments, budget scenarios and the decisions they feed. Other model families join through family profiles (section 16) without changes to the core.

In scope for v1

  • MMMs from in-house frameworks, modeling-api, Meridian, Robyn, PyMC-Marketing and vendors
  • Vendor response curves as parameters or spend tables
  • Test plans and results for geo, user-level holdout and ghost-ad designs, including plans imported from Google’s geo planning product
  • Scenarios, optimisations and decision records
  • Lineage of every dataset and change feeding or leaving our own models
  • The audience family profile, at the Sandbox and Pilot lifecycle stages only

Non-goals. UMP does not fit models or prescribe how to fit them. It does not replace Meridian, Robyn or vendor tools, and it is not a dashboard. It does not make decisions: machines draft, named people approve.

Conformance classes. A tool claims one or more classes and proves each against the conformance suite (section 21).

ClassMust doProved by
ProducerWrite valid artifacts with full estimand, representation and provenance fieldsSchema and round-trip fixtures
ConsumerApply the fallback a gate prescribes (or stop, in strict mode); show trust and fallback labelsExpected-error fixtures
Engine adapterAnswer standard requests it declares, with typed errors for the restScore, optimise and calibrate fixtures with known answers
Gate engineEvaluate published policies on every operation and transition each gate’s applies_to namesPolicy unit tests in the suite
Agent hostPrefer validated artifacts, write drafts only, cite or label a best guessGolden-question and injection fixtures (section 17)
Log verifierVerify records, checkpoints and inclusion and consistency proofs of the integrity log independentlySection 18 test vectors
Hosted endpointAnswer the standard requests a Tier X model declares over HTTP, with typed errors for the restEngine profile fixtures (section 8) run over HTTP

Class requirements. Each class is bound by the requirements tagged with it in the conformance checklist published with the suite (section 21). The minimum requirements include:

  • REQ-02-01 (Consumer): a Consumer MUST preserve every fallbacks entry it receives and MUST re-emit it with any result it derives or forwards.
  • REQ-02-02 (Consumer): a Consumer MUST NOT present a derived result whose input fallbacks it dropped.
  • REQ-02-03 (Consumer with a user interface): it MUST show the trust label and every fallback label beside the number.
  • REQ-02-04 (Gate engine): it MUST evaluate each gate on every operation and transition named in that gate’s applies_to.
  • REQ-02-05 (Log verifier): it MUST reproduce every section 18 test vector, check every checkpoint’s consistency proof, and raise integrity.broken on failure.

Conformance claim. A claim MUST be a machine-readable document giving the class, spec version, suite version, threshold-set version, the standard requests and model tiers covered, and the SHA-256 digest of the suite report. A partial claim MUST list what it covers. Anything unlisted is unclaimed, not conformant.

Design rules (rules 8 and 9 extend the original seven):

  1. When the best answer is unavailable, use the next best and say which was used.
  2. Each model declares what it can calculate; tools check before asking.
  3. Uncertainty travels with every result; summaries are labelled as summaries.
  4. Never overwrite; a new version points to its parent.
  5. Media names match through a maintained lookup, never by similarity.
  6. Each artifact states how far it can be audited and claims no more.
  7. Independent of language and storage.
  8. A number travels with its estimand; any part that is unknown is stated as an assumption and widens the range. Schema-valid numbers can still answer different questions, so meaning travels with the value (section 1).
  9. Gates degrade before they block. Missing or weaker evidence leads to a labelled best guess; work stops only where going ahead is unsafe. A blocked step pushes the work into unlabelled spreadsheets, where no gate sees it at all.