mapi_spec¶
This package has contracts that use only Pydantic. It does not use numpy, xarray or pandas, because the control plane depends on this package.
Data schema¶
Data contract primitives: dims, variables, ragged structure and dataset schemas.
A :class:DataSchema describes an xr.Dataset without any arrays: its dims (each with a
:class:DimPolicy saying what may happen to it at score time), its variables (with a
:class:Role, dtype, units and missingness) and its ragged structure.
A :class:DataSchemaTemplate is a static JSON function of enumerated settings: plugins
export one per model so the control plane can compute the required input schema for given
settings without running plugin code.
Bases: StrictModel
A dimension and its score-time policy.
Bases: StrEnum
What a variable means to a model.
Bases: StrictModel
A data variable: its dims (by name), dtype, role, units and missingness.
Bases: StrictModel
Ragged nesting (hh → trip): long form (obs + index vars) or padded + mask.
Bases: StrictModel
Schema of an xr.Dataset: dims, variables, layout and ragged structure.
vars_with_role(*roles: Role) -> list[str]
Variable names whose role is one of roles (declaration order).
renamed(mapping: Mapping[str, str]) -> DataSchema
Rename dims and/or variables (old -> new), updating every reference.
Bases: StrictModel
A DataSchema-shaped JSON template plus the settings parameters it depends on.
Resolution is pure and static: resolve(settings) extracts each declared parameter
from the settings (falling back to its default), checks its type/enum, renders the
:mod:~mapi_spec.template placeholders, deep-merges every matching case and
validates the result as a :class:DataSchema.
from_schema(schema: DataSchema) -> DataSchemaTemplate
A constant template (no parameters).
Extract and type-check every declared parameter from settings.
Render to plain JSON (not yet validated as a DataSchema).
resolve(
settings: Mapping[str, Any] | None = None,
) -> DataSchema
Resolve to a :class:DataSchema (raises TemplateError or ValidationError).
try_resolve(
settings: Mapping[str, Any] | None = None,
) -> tuple[DataSchema | None, str | None]
Like :meth:resolve but returns (None, message) instead of raising.
Documents¶
Declarative documents: ModelSpec, ScoreRequest and the wrapper kinds.
Every document has apiVersion: mapi/v1 and a kind discriminator. All models forbid
unknown fields. See :mod:mapi_spec.loader for strict parsing and canonical hashing and
:mod:mapi_spec.validate for static validation against a catalog.
Bases: _Document
The declarative modelling file.
Bases: StrictModel
The dataset, side tables, binding to canonical names and the split.
For a tree dataset (M4), nodes picks the nodes whose variables are merged into the
ONE flat dataset the pipeline sees (:meth:mapi_spec.tree.TreeSchema.flat), aligned per
join. Without nodes a tree dataset's root node is read (when it has variables).
nodes on a plain dataset is an error. The binding applies to the merged names.
For an image collection (v0.15), collection says how a model that reads datasets
sees it (a seeded sample); a model that trains on batches reads every image.
Bases: StrictModel
Canonical name → dataset name, for dims and variables (geo: market).
Names that are not bound keep their dataset name. The binding is stored in the bundle and reversed on every output by default.
Bases: StrictModel
A small constant dataset (FX rates, holidays ...) used by steps.
A side table that is a tree dataset (M4) reads ONE node, named by node.
Bases: StrictModel
One pipeline step: {id, use: core.resample@^1, with: {...settings}}.
Bases: StrictModel
The model plugin, its settings and fit configuration.
settings.dim_policies is platform-reserved: a mapping dim → policy that may only
tighten the plugin's policies. It is removed before plugin settings validation.
Settings without platform-reserved keys (what the plugin schema validates).
Bases: StrictModel
Platform fit settings. Plugin-specific limits come from the manifest's fit_schema.
Bases: _Document
Score a fitted model with replayed preprocessing.
Bases: StrictModel
Bases: StrictModel
Change var on the coords selected by sel: exactly one of multiply/add/set.
Loading and hashing¶
Strict YAML 1.2 / JSON loading, apiVersion upgrades, canonical JSON and spec hashing.
The loader is safe for hostile input:
- size ≤ 256 KB, nesting depth ≤ 32, ≤ 10 000 nodes (all configurable via :class:
Limits); - YAML anchors, aliases, explicit tags (
!foo,!!python/...),%TAGdirectives and multiple documents are rejected; - plain scalars are resolved with the YAML 1.2 core schema (
yes/on/0777/dates stay strings), NaN/Infinity are rejected, duplicate keys are rejected and every mapping key must be a string; - JSON input is parsed with the same limits.
:func:load_document returns a typed document (discriminated on kind) after running the
apiVersion upgrade chain (the API accepts N and N-1). :func:canonical_json and
:func:spec_hash give a normalised, deterministic serialisation for immutable revisions.
Limits(
max_bytes: int = 256 * 1024,
max_depth: int = 32,
max_nodes: int = 10000,
max_scalar_bytes: int = 64 * 1024,
)
Parser limits for untrusted documents.
load_document(
text: str | bytes, limits: Limits = DEFAULT_LIMITS
) -> Any
Parse YAML/JSON text and validate it as a typed document.
load_score_request(
text: str | bytes, limits: Limits = DEFAULT_LIMITS
) -> ScoreRequest
Validate a parsed mapping as a typed document (ModelSpec, ScoreRequest ...).
Deterministic JSON: defaults omitted, keys sorted, no whitespace, UTF-8, ints for integral floats. Equal documents (after normalisation) produce identical text.
sha256:<hex> of the canonical JSON (the immutable revision identity).
Capabilities, manifests and outputs¶
Model capabilities, uncertainty representations and the model manifest.
The :class:Representation of a model's output decides which aggregations and summaries
are statistically valid. Those rules live here as data (:data:REPRESENTATION_RULES) and
small functions (:func:can_aggregate, :func:summary_allowed) so the control plane, the
runner summariser and the UI all agree.
Bases: StrEnum
How a model encodes uncertainty in its outputs.
(quantile, *dims) marginal quantiles — no cross-cell aggregation.
Bases: StrictModel
What a model plugin can do (declared in its manifest).
Bases: StrictModel
Everything the control plane knows about a model plugin version (static JSON).
Bases: StrictModel
An output variable the model writes (e.g. y_hat).
full_dims(schema: DataSchema | None) -> list[str]
dims plus the dims of dims_of in the model's resolved input schema.
Bases: StrictModel
A derived quantity (contributions, response curves, ...).
Bases: StrictModel
What a score request returns.
summary_for(rep: Representation) -> list[str]
The requested summary, or the representation default if none was given.
Bases: StrictModel
Reduce outputs over dims before summarising (validity depends on representation).
Bases: StrictModel
A diagnostic finding that rolls up into the sanity scorecard.
can_aggregate(
rep: Representation,
over_dims: list[str] | tuple[str, ...],
how: str = "sum",
) -> AggregationVerdict
Is reducing an output over over_dims with how statistically valid?
summary_allowed(rep: Representation, name: str) -> bool
Whether summary name can be computed from output in representation rep.
default_summary(rep: Representation) -> list[str]
The default summary list for a representation.
Parse a summary name. Raises ValueError for unknown names.
Schema effects and stages¶
Step kinds, SchemaEffects, context needs and static schema propagation.
A :class:SchemaEffect is a data-free description of how a preprocessing step changes a
:class:~mapi_spec.schema.DataSchema. Step authors export it as JSON (per image digest);
the control plane evaluates it with :func:propagate and never imports step code.
Effects may be templates: strings can reference the step's settings with the
:mod:~mapi_spec.template placeholder syntax ("{dim}", "{*vars as v}"). Every
referenced setting must be listed in template_params. :func:resolve_effect renders a
template with concrete settings into a plain :class:SchemaEffect.
Within one effect, operations apply in this order:
renames → drops → transforms_dims → stacks → unstacks → updates → derives → adds.
Bases: StrEnum
RESHAPE changes structure (not invertible past it); VALUE rescales; FEATURE adds.
Bases: StrictModel
How a step changes the schema. See the module docstring for ordering rules.
Bases: StrictModel
Per-group tail a step needs at score time: n rows along along within groups.
min_lag is the smallest lag the step applies (for recursive-forecast checks);
it defaults to 1 when the step depends on the target.
Stage(name: str, schema: DataSchema | None)
The dataset schema after a step (name = step id; raw before any step).
propagate(
raw_schema: DataSchema,
steps: Sequence[
tuple[str, EffectInput, Mapping[str, Any]]
],
*,
pointer_base: str = "/pipeline",
pointers: Sequence[str] | None = None,
) -> PropagationResult
Propagate raw_schema through steps = [(step_id, effect, settings), ...].
Effects may be templates; settings are substituted into them. A dynamic effect
stops static propagation: that stage and every later one are None and a
SCHEMA_DYNAMIC warning is reported (an explicit dry-run can resolve it).
resolve_effect(
raw: SchemaEffect | Mapping[str, JsonValue],
settings: Mapping[str, Any],
) -> SchemaEffect
Render a templated effect with concrete settings.
Raises :class:~mapi_spec.template.TemplateError or pydantic.ValidationError.
apply_effect(
schema: DataSchema, effect: SchemaEffect
) -> tuple[DataSchema | None, list[str]]
Apply one resolved effect. Returns the new schema (or None) and problems.
Static validation¶
Static (JSON-only) validation of ModelSpecs and ScoreRequests.
:func:validate_spec checks a :class:~mapi_spec.spec.ModelSpec against a
:class:~mapi_spec.catalog.Catalog and the dataset's raw :class:DataSchema:
- binding references exist and do not collide;
- every
use:resolves to a non-yanked version (semver range); - step and model settings validate against their JSON Schemas;
- side-table references exist;
- the schema is propagated through every step's SchemaEffect, with per-step input checks;
- the split stage is resolved (default: end of the stateless prefix), its dims exist there,
and
LEAKAGE_RISKis raised if a stateful step precedes it; - the model's schema template is resolved and compared with the final stage; spec dim policies may only tighten the plugin's;
- recursive forecasting (a target-dependent lag feeding a longer holdout) is rejected unless the model owns recursion; outputs, plots, metrics and resource class are checked;
- (v0.15) the model's side tables exist; an image collection dataset takes no pipeline steps for a model that trains on batches and only row splits; GPU class and epoch checkpoints match the model.
:func:validate_score_request runs the static CoordPlan checks for a ScoreRequest from
metadata only (resolved spec + fit coordinate summaries + dim policies).
Every finding is an :class:~mapi_spec.errors.Issue with a JSON pointer into the document.
SpecValidation(
issues: list[Issue],
stages: list[Stage],
resolved: ResolvedSpec | None = None,
)
Outcome of :func:validate_spec.
Bases: StrictModel
What the control plane stores with a run: everything score-time checks need.
Bases: StrictModel
Per-model-version metadata used for static CoordPlan checks.
fit_coords maps a stage name to coordinate summaries of the full fit data at that
stage (the production refit), at least for the model stage.
validate_spec(
spec: ModelSpec,
catalog: Catalog,
raw_schema: DataSchema,
*,
tree_schema: TreeSchema | None = None,
dataset_layout: DataLayout = "dataset",
side_table_layouts: Mapping[str, DataLayout]
| None = None,
) -> SpecValidation
Statically validate spec against catalog and the dataset's raw_schema.
For a tree dataset (M4) pass its tree_schema too (raw_schema is then its root
node's schema and is not used): the pipeline input is resolved from data.nodes by
:func:resolve_input_schema. For an image collection (v0.15) pass
dataset_layout="collection" (raw_schema is the collection's schema, with the
virtual pixels variable); side_table_layouts (name → layout) lets the check refuse
side tables that are collections.
validate_score_request(
req: ScoreRequest,
meta: ScoreMetadata,
inputs: DataSummary | None = None,
*,
max_draws: int | None = None,
live_capabilities: Capabilities | None = None,
) -> list[Issue]
Static CoordPlan checks for req against a fitted model's metadata.
inputs summarises the payload (data or scenario) as it enters at
req.inputs_stage; dataset names are mapped through the binding at every stage, as
the runtime does. When it says where the targets hold values (inputs.observed), a
forecast's actuals are checked (:mod:mapi_spec.actuals). live_capabilities are
those of the plugin build that will score, which may have gained score-time features
(conditions_on_actuals) since the fit; the fitted ones are used otherwise.
Catalogs¶
Static catalog entries for steps, models and plots (exported per image digest).
The control plane validates specs against these JSON shapes only — it never imports the step or model code that produced them.
Bases: StrictModel
A set of catalog entries with semver resolution helpers.
resolve_step(ref: UseRef | str) -> StepCatalogEntry | None
Highest non-yanked step version satisfying the ref's range.
Bases: StrictModel
Everything static validation needs to know about one step version.
effect and context are JSON templates over the step's settings (see
:mod:mapi_spec.template). var_params/dim_params name settings fields whose
values are variable/dim names that must exist at the step's input stage, and
side_table_params name fields that refer to data.side_tables entries.
Bases: StrictModel
A plot recipe the runner can produce.
catalog_about_issues(
catalog: Catalog,
*,
extra_ids: Iterable[str] = (),
builtins: bool = True,
) -> list[Issue]
The catalog rule (M3.5): an ABOUT_INCOMPLETE error for every non-yanked step
and model of catalog whose About is missing or incomplete
(:func:mapi_spec.about.about_issues), and a warning for each related id that
names no known item, located by JSON pointers into the catalog document.
Known ids are the catalog's own, extra_ids and (builtins) every built-in card's id
(:func:mapi_spec.abouts.builtin_abouts: splits, templates, workflow nodes, checks).
Errors¶
Stable error codes, RFC 7807 problem documents and the MapiError hierarchy.
Every error the platform returns carries a stable machine-readable code (one of
:class:ErrorCode). Codes never change meaning once released; new codes are additive.
Untrusted text (tracebacks, user-supplied values echoed back, plugin messages) is always
carried in a separate untrusted_text field and never merged into detail/message.
Bases: StrEnum
Stable error codes. See :data:ERROR_INFO for HTTP status and titles.
MapiError(
detail: str,
*,
code: ErrorCode | None = None,
pointer: str = "",
untrusted_text: str | None = None,
issues: Sequence[Issue] = (),
)
Bases: Exception
Base exception: carries a stable code, a JSON pointer and optional untrusted text.
Bases: BaseModel
One problem found while validating a document, located by a JSON pointer.
Bases: BaseModel
RFC 7807 problem document with mapi extensions (code, errors, untrusted_text).
Bases: StrEnum
How bad an :class:Issue is. Only error blocks submission.
ValidationFailed(
issues: Sequence[Issue], detail: str | None = None
)
Bases: SpecError
Static validation produced at least one error; issues holds them all.
http_status(code: ErrorCode | str) -> int
HTTP status for a code (500 for unknown codes).
Build an RFC 6901 JSON pointer from path segments ("" is the whole document).
Plot envelope¶
The PlotSpec envelope: a Vega-Lite or Plotly spec plus named datasets and provenance.
Datasets are referenced by name only (Vega-Lite data: {name: ...}, Plotly
{"$ref": "name.column"}) and never by URL. The full plot sanitiser (schema validation,
HTML stripping, URL rejection everywhere) lives in mapi_plots; this module enforces the
envelope contract: shapes, name references and the no-URL data rule.
Bases: StrictModel
A renderable plot. spec is Vega-Lite JSON or Plotly {data, layout, ...}.
Bases: StrictModel
A small Arrow IPC stream embedded as base64.
Bases: StrictModel
A dataset stored next to the bundle (path relative to the bundle root).
Bases: StrictModel
Role/dim template bindings, e.g. {x: "dim:time", y: "role:target"}.
Templates bound this way apply to every model of a family, not one bundle.
Bases: StrictModel
Who produced a plot: exactly one of recipe, plugin or author.
Executor protocol¶
mapi_spec.run exports the executor shapes again with no changes.
Executor protocol shapes (plan §8.4), shared by the control plane and the runtimes.
The control plane drives every backend (in-process, local process, docker, Cloud Run Jobs,
...) through one :class:Executor protocol. Identifiers travel in the environment only; the
executor's status API is the liveness source of truth and events are advisory progress.
These are pure Pydantic shapes: no numpy, xarray or plugin code.
Bases: Protocol
A backend driver. Identifiers reach the run through the environment only.
Bases: StrictModel
One attempt of a run, as handed to an executor (JSON only).
Bases: StrEnum
What a run does. validate is the explicit sandboxed dry-run of a spec.
sweep, backtest, sbc and campaign are fan-out parents (plan §6.2,
M4): they own child runs and never execute on an executor themselves.
M3: business priors → model-space priors plus a prior-predictive preview in business
units (sandboxed; :class:mapi_spec.calibration.PriorTranslationJob).
M3.5: a workflow run (a parent: the orchestrator walks the DAG and starts
workflow_node runs and child fits; :mod:mapi_spec.workflow).
M3.5: one sandbox node of a workflow (:class:mapi_spec.workflow.WorkflowNodeJob).
M3.5: Data or Model Explorer views (:class:mapi_spec.explore.ExploreJob).
M3.5: the governance checks of one model version in the sandbox
(:class:mapi_spec.governance.GovernanceJob; output governance.json).
M4: list the groups and arrays of an upload from its metadata (resource class S, short
timeout, no network; :class:mapi_spec.jobs.InspectJob, output inspection.json).
Bases: StrictModel
Executor-side identity of a submitted attempt.
Bases: StrEnum
Bases: StrictModel
Executor-reported status of an attempt (the liveness source of truth).
Bases: StrictModel
What an executor can do; placement rules are enforced against these.
Public tenants need isolation >= gVisor with egress and DNS enforced.
Bases: IntEnum
How strongly a backend isolates a run (ordered: higher is stronger).
Bases: StrictModel
One progress event; (run_id, attempt, seq) is the dedupe key.
Bases: StrEnum
Event types written by the supervisor (see :mod:mapi_spec.run for payloads).
metrics, params and tags (v0.11) and media (v0.14) carry run tracking
data: the control plane writes them to the tracking tables, not to the stored event list.
Run protocol¶
For the full description, read Run protocol.
The run protocol (plan §8.1, §8.3–§8.7): state machine, bootstrap, events, manifests.
Everything a control plane, an executor and a runner must agree on lives here, as pure Pydantic / stdlib code:
- :class:
RunState, :class:RunTriggerand the transition table :data:TRANSITIONS, with the pure function :func:transitionused by the API, dispatcher, events service, reconciler, finalizer and tests; - :class:
RunSpec— the bootstrap object written tobootstrap/<nonce>before an execution starts (signed URLs, a per-run HMAC key, limits, deadline); - :class:
~mapi_spec.executor.RunEventpayload models, canonical HMAC-SHA256 signing and verification, and the stdout JSONL wire format ({"mapi_event": {...}}per line); - :class:
OutputManifest(the runner's claim), metadata-only verification against server-computed hashes, and the platform-signed :class:BundleManifest; - the runner invocation contract: environment (:class:
RunnerEnv) and exit codes (:class:RunnerExit).
Executor-facing shapes (:class:Isolation, :class:ExecutorCaps, :class:Handle,
:class:ExecStatus, :class:RunKind, :class:RunEvent, ...) are defined in
:mod:mapi_spec.executor and re-exported here unchanged, so both import paths name the
same classes.
Bases: StrEnum
Lifecycle of a run (plan §8.3). Terminal states never change again, except that a
PARTIAL run can be resumed (RESUME → QUEUED with a new attempt).
Bases: StrEnum
Why a run ended FAILED; each maps to one stable :class:ErrorCode.
from_error_code(code: ErrorCode | str) -> FailureClass
Inverse of :attr:error_code; unknown codes are infra.
Bases: StrEnum
What happened to a run. The owner of each transition is in :data:TRANSITIONS.
Bases: StrEnum
Component allowed to apply a transition (plan §8.3 'who owns which transitions').
TransitionRule(
source: RunState,
trigger: RunTrigger,
target: RunState,
actors: frozenset[Actor],
failure_class: FailureClass | None = None,
new_attempt: bool = False,
)
IllegalTransition(state: RunState, trigger: RunTrigger)
Bases: StrictModel
The bootstrap object at bootstrap/<nonce> (plan §8.1).
Written by the dispatcher right before the executor call and deleted by the supervisor
immediately after reading it. It is the only way a run receives URLs and its HMAC key:
the environment carries just :class:RunnerEnv.
JSON for logs: the HMAC key and every signed URL are replaced by "<redacted>".
Bases: StrictModel
The entire configuration an execution receives through its environment.
MAPI_RUN_ID/MAPI_NONCE— per execution (the only executor overrides);MAPI_BOOTSTRAP_URL— static per executor/Job: a template containing{nonce}, e.g.gs://mapi-prod-bootstrap/{nonce}(cloud: read with the execution service account) orhttp://127.0.0.1:4443/storage/v1/b/mapi-local-bootstrap/o/{nonce}(local GCS shim);MAPI_EXEC_TOKEN— local only: bearer token of the shim's exec principal (get+delete on the bootstrap bucket). Never set in the cloud.
Bases: IntEnum
Supervisor exit codes. 0 only after the manifest upload and the complete event.
137 and 143 are not chosen by the supervisor: they are SIGKILL (OOM killer or hard timeout) and an unhandled SIGTERM as reported by the executor.
Bases: StrEnum
Coarse phase reported by the runner (drives progress text in the UI).
Bases: _Payload
state: the runner entered a new phase.
Bases: _Payload
progress: coalesced to at most one event per limits.progress_interval_s.
Bases: StrictModel
Per-chain sampler progress (Expert run view).
Bases: _Payload
log: one captured worker output line (ANSI/control sequences already stripped).
Stream user (v0.11) is a structured line the model code wrote through the tracking
API; only those lines carry a level and fields.
Bases: _Payload
warning: a non-fatal, platform-classified problem.
Bases: _Payload
checkpoint: a checkpoint object was uploaded under the checkpoint prefix.
Bases: _Payload
metric: a scalar metric (holdout error, elpd, ...).
Bases: _Payload
complete: outputs and the manifest are uploaded; the supervisor exits next.
Bases: _Payload
error: the run failed; message is platform-authored, plugin text is untrusted.
InvalidEvent(
detail: str,
*,
code: ErrorCode | None = None,
pointer: str = "",
untrusted_text: str | None = None,
issues: Sequence[Issue] = (),
)
Bases: SandboxViolation
A line claimed to be an event but is malformed, oversized, or fails its HMAC.
Bases: StrictModel
The runner's manifest at {prefix}manifest.json: a claim, verified at FINALIZING.
objects maps a relative output name (bundle.zarr.zip, ui/summaries.json) to
the uploaded object; every key must equal prefix + name. The supervisor signs it with
the run HMAC key (hmac), which only proves it came from this run's supervisor.
Bases: StrictModel
One uploaded object as claimed by the runner (hashes computed while streaming).
Bases: StrictModel
What the runner actually read for one input (copied from the RunSpec, then checked).
VerificationResult(
issues: list[Issue], verified: dict[str, ObjectMeta]
)
Outcome of :func:verify_output_objects.
verified: dict[str, ObjectMeta]
Relative name → observed metadata (pin generation when copying).
Bases: StrictModel
The platform-signed manifest of a bundle (manifest.json + manifest.sig).
Binds tenant, run, base digest, overlay hash, spec hash, data hash and every destination generation. URLs to a bundle are minted only after this signature checks.
Bases: StrictModel
A verified object after the server-side copy into the bundles bucket.
Bases: StrictModel
Platform signature over :func:bundle_manifest_signing_bytes.
HS256 (HMAC-SHA256 with a server-held key) is the local/M1 signer; KMS asymmetric
signers (M2) use their own alg and the same signing bytes.
transition(
state: RunState | str,
event: RunTrigger | str,
*,
actor: Actor | str | None = None,
) -> TransitionRule
Apply event to state; return the matching rule or raise :class:IllegalTransition.
Pure and deterministic. Callers persist rule.target (and failure_class; bump the
attempt when new_attempt) with a compare-and-set on the old state. If actor is
given it must be one of the rule's owners.
failure_class_for_exit(
exit_code: int,
) -> FailureClass | None
Classify a supervisor exit code (None = not a failure: OK or PARTIAL).
Unknown non-zero codes (e.g. 1 from a crashed supervisor) are infra. A run that
the user cancelled is handled by the CANCELLING state, not by this function.
The scrubbed environment for the untrusted worker process.
parse_payload(event: RunEvent) -> BaseModel
Validate event.payload against the model for event.type.
make_event(
run_id: str,
attempt: int,
seq: int,
payload: BaseModel,
*,
ts: datetime | None = None,
key: bytes | str | None = None,
) -> RunEvent
Build a :class:RunEvent from a typed payload (type inferred), signed if key.
Canonical UTF-8 JSON used for every signature in the run protocol.
Models are dumped in JSON mode with defaults included (so signer and verifier never
disagree about defaulted fields), exclude top-level keys are dropped, keys are sorted,
there is no whitespace, and integral floats become ints (as :func:canonical_value).
Return a copy of event with hmac set (the supervisor does this per line).
verify_event(event: RunEvent, key: bytes | str) -> bool
Constant-time check of event.hmac; False if it is missing or wrong.
compute_event_hmac(
event: RunEvent, key: bytes | str
) -> str
Lower-case hex HMAC-SHA256 of :func:event_signing_bytes under the run key.
encode_event_line(event: RunEvent) -> str
One JSONL line (without the trailing newline) exactly as the supervisor prints it.
decode_event_line(line: str | bytes) -> RunEvent | None
Parse one stdout line. Non-JSON lines and JSON without mapi_event return None
(they are not events); a present-but-invalid event raises :class:InvalidEvent.
event_from_log_payload(
json_payload: Mapping[str, Any],
) -> RunEvent
Parse a Cloud Logging jsonPayload (or a decoded stdout line) into a RunEvent.
verify_output_objects(
manifest: OutputManifest,
observed: Iterable[ObjectMeta],
*,
expected_prefix: str,
allowed_extra_prefixes: Iterable[str] = (
"checkpoints/",
"media/",
),
) -> VerificationResult
Metadata-only FINALIZING check (plan §8.6), with no object content read.
observed is the store listing of expected_prefix (server-computed size, CRC32C,
MD5, generation). Every claimed object must exist with identical size and hashes, and
no unexpected object may exist under the prefix except manifest.json and the
allowed_extra_prefixes (relative; checkpoints and, since v0.14, logged media). Each
discrepancy is an INTEGRITY_FAILURE.
sign_output_manifest(
manifest: OutputManifest, key: bytes | str
) -> OutputManifest
verify_output_manifest(
manifest: OutputManifest, key: bytes | str
) -> bool
sign_bundle_manifest(
manifest: BundleManifest, key: bytes | str, key_id: str
) -> BundleManifest
HS256-sign a bundle manifest (the local/M1 signer).
verify_bundle_manifest(
manifest: BundleManifest,
keys: Mapping[str, bytes | str],
) -> bool
Check an HS256 signature against keys (key id → key). KMS algs are verified by the
KMS signer in the server and raise NotImplementedError here.
catalog_digest(catalog: Catalog) -> str
sha256:<hex> identity of a catalog snapshot (pinned in every RunSpec).
Decode a base64 HMAC key; it must hold at least 32 bytes.
Job payloads¶
See Job payloads and object names.
Run job payloads: what RunSpec.spec holds for each run kind, plus the input and
output object names shared by the control plane (which writes RunSpecs and reads outputs)
and the runner (which reads RunSpecs and writes outputs).
========== ============================================ ======================================
kind RunSpec.spec inputs (RunSpec.inputs names)
========== ============================================ ======================================
ingest :class:IngestJob (a bare IngestMapping upload (the raw file; dataset
is also accepted) is accepted as a fallback); M4
assemblies: upload.<i> per source
inspect :class:InspectJob (M4) upload; output inspection.json
describe :class:DescribeJob none
validate a ModelSpec document dataset, optional schema,
side.<name>
fit a ModelSpec document dataset, optional schema,
side.<name>
score a ScoreRequest document, or bundle, optional
:class:ScoreJob bundle.manifest, dataset (the
request's data), scenario,
side.<name>, history (M3: the
fit dataset, for carry_last /
seasonal_naive future covariates)
evaluate :class:EvaluateJob bundle, dataset, ...
derive :class:DeriveJob bundle, dataset, side.<name>
render :class:RenderJob plots (a plots.json)
translate_ :class:~mapi_spec.calibration.Prior- ``dataset``, optional ``schema``,
priors TranslationJob (M3) side.<name>; output priors.json
workflow_ :class:~mapi_spec.workflow.WorkflowNode- ``in.<port>.<k>`` (each bound input);
node Job (M3.5) output node_result.json
explore :class:~mapi_spec.explore.ExploreJob dataset (+ schema) or
(M3.5) bundle, bundle.manifest,
compare.<k>; output ui/explore/
optimize :class:~mapi_spec.optimize.OptimizeJob bundle, bundle.manifest; output
(M3.5) optimize.json
governance :class:~mapi_spec.governance.Governance- ``bundle``, ``bundle.manifest``,
Job (M3.5) dataset, optional evaluation
(+ evaluation.manifest); output
governance.json
========== ============================================ ======================================
workflow runs are parents (the orchestrator walks the DAG) and never execute.
Datasets are single-object *.zarr.zip files (M4: a tree dataset is a data.tree.zip
with a tree_schema.json; see :mod:mapi_spec.tree). A dataset's raw schema comes from the
schema input, else the zarr root attribute mapi_schema (written by ingest), else it is
inferred. bundle is the bundle.zarr.zip of a fit and bundle.manifest the SDK
bundle manifest that the fit uploaded as bundle.manifest.json.
This module is pure Pydantic (no numpy/xarray): the control plane imports it.
ScoreRequest dataset fields → the input name the runner reads them from.
Root attribute of a *.zarr.zip dataset holding its raw DataSchema as JSON. Ingest
writes it; uploads of self-describing datasets may carry it (dim policies, roles, units) —
otherwise ingest infers a schema (every dim fixed, every variable a covariate).
What an ingest run uploads: the data, its DataSchema, its DataSummary (used for
CoordPlan checks of score payloads), the profile, the UI preview and (M3) columns.json:
one :class:~mapi_spec.guided.ColumnProfile per column of the uploaded table (for MFF
uploads: per pivoted VariableName), which Guided role detection reads.
JSON sidecars of a fit: list[Diagnostic], :class:MetricsSidecar, list[PlotSpec]
and the fit coordinates {stage: {dim: CoordSummary}} (ScoreMetadata.fit_coords).
The SDK bundle manifest (content hashes, lineage). manifest.json is reserved for the
runner's :class:~mapi_spec.run.OutputManifest.
Bases: StrictModel
An ingest run: an explicit column mapping, the MFF preset, an assembly of uploads (M4)
or an image collection (v0.15, :class:~mapi_spec.collection.CollectionIngest).
From M3 the mapping itself may name a preset (mapping.preset: mff or
meridian, with its options in the mapping); the top-level
preset/mff pair stays for v0.2 callers.
Bases: StrictModel
What the uploader declared about the raw file (informational; never trusted).
Bases: StrictModel
The MFF ("model feature format") long-table preset.
One row per (period, geography, product, campaign, outlet, creative, variable) with
the value in value_column. Period becomes the time dim; VariableName values
become data variables. The rest follows mmm-framework's MFF semantics (its
MFFLoader), so both read a file the same way:
GeographyandProductbecome dims when they hold more than one level. A blank label (orNational,Total,All) is the coarser level: a variable given only there (national TV) is spread across the levels - equal weights, or the KPI's share of each level (allocation; mmm-framework's defaults: geographies equally, products by sales). A variable may not mix the two levels.Campaign,OutletandCreativeare breakdowns: rows that differ only there are summed, unless the column is listed insplit(then it becomes a dim).
combine joins several key columns into one dim ({"geo": ["Geography", "Product"]}
gives labels such as US|Watches) - a model with one panel dim can then fit a
geography x product panel. A combined dim is created even when it has a single label,
so {"geo": ["Geography"]} keeps a national table's geo dim.
Bases: StrictModel
A describe run exports the image's catalog (catalog.json).
Bases: StrictModel
A score run: the request plus runner-side options.
Bases: StrictModel
Evaluate a bundle on dataset (default request: the fit-time evaluation request;
default data: the dataset version the model was fitted on).
Bases: StrictModel
Compute declared derived quantities of a bundle over the dataset input.
Bases: StrictModel
Render plots from a plots.json to static images (renders/<plot id>.<format>).
pdf (M3) and an explicit width/height in CSS pixels (before scale) are
for reports; Vega-Lite renders with vl-convert, Plotly with kaleido, in the sandbox.
Bases: BaseModel
metrics.json as the control plane reads it: holdout metric values (None when a
metric could not be computed) and why metrics were skipped. Unknown fields are ignored so
the SDK's report can grow.
parse_job(kind: RunKind, spec: dict[str, Any]) -> Job
Parse RunSpec.spec for kind (raises MapiError(SCHEMA_INVALID)).
parse_model_job_params(
kind: RunKind, params: dict[str, Any]
) -> EvaluateJob | DeriveJob | RenderJob
Validate the params of an evaluate/derive/render run request.
The input name of side table name (side.<name>).
UI views¶
Precomputed UI views: small JSON documents the runner writes next to datasets and bundles and the browser reads through signed URLs (the control plane never decodes them).
Every view carries a format tag so readers can refuse versions they do not understand:
===================== ======================== ==========================================
file format content
===================== ======================== ==========================================
preview.json mapi.preview/v1 a long-format sample: dim then var columns
profile.json mapi.profile/v1 per-variable statistics + missingness grid
ui/tree_index.json mapi.tree-index/v1 structure of the bundle DataTree (no arrays)
ui/preprocess.json mapi.preprocess/v1 learned preprocessing state per pipeline step
ui/grids/mapi.response-grid/v1per-draw response curves for what-if sliders
response_curves.json (M3; :class:ResponseGridView)
ui/guided/mapi.forecast/v1a forecast score run's KPI: history, then the
forecast.json forecast fan after the split (M3)
ui/guided/mapi.counterfactual/v1a counterfactual score run's change of the
counterfactual.json KPI, in total and per period (M3)
ui/explore/mapi.model-explorer. quantised posterior rasters of the outputs
fields.jsonfields/v1 that declare a grid (:class:FieldsView)
===================== ======================== ==========================================
Views are produced inside the sandbox from untrusted data: strings are capped and readers must render them as text only.
Bases: _View
preview.json: the first rows of the dataset in long format.
Bases: _View
Bases: _View
profile.json: per-variable statistics and a missingness heatmap.
Bases: _View
Bases: _View
Fraction missing per (variable, coordinate of the time-like dim along).
Bases: _View
ui/tree_index.json: one entry per DataTree node (keyed by its path).
Bases: _View
Bases: _View
Bases: _View
Object storage¶
Object storage contracts (plan §8.6): metadata, the ObjectStore protocol, signed URLs,
integrity hashes and the bucket/key naming scheme.
The control plane talks to object storage only through :class:ObjectStore. Two
implementations exist outside this package: ShimObjectStore (the local GCS emulator in
mapi_gcs_shim) and GcsObjectStore (google-cloud-storage). Both expose GCS semantics:
- every write creates a new generation;
if_generation_match=0means "only if the object does not exist" (write-once); - the server computes CRC32C (base64 of the big-endian 4-byte value) and MD5 (base64 of the 16-byte digest) — uploader-supplied hashes are never evidence;
- signed GET URLs are pinned to a generation and honour HTTP
Range; signed PUT URLs and V4 POST policies (key prefix +content-length-range) are how sandboxes write, so the executor never holds storage credentials.
Object keys are always built server-side from validated IDs with the helpers at the bottom of this module (never from user-supplied paths).
Bases: Protocol
Async object-store driver used by the control plane (GCS semantics).
Conventions shared by every method:
generationpins the source generation to read (None= live generation);if_generation_matchis a precondition on the target:0means "must not exist",nmeans "live generation must ben"; a failed precondition raises :class:PreconditionFailed;- missing buckets/objects raise :class:
NotFound; denied operations raise :class:AccessDenied.
put(
bucket: str,
name: str,
data: bytes,
*,
content_type: str | None = None,
metadata: Mapping[str, str] | None = None,
if_generation_match: int | None = None,
) -> ObjectMeta
Upload data as a new generation and return its server-computed metadata.
get(
bucket: str,
name: str,
*,
generation: int | None = None,
byte_range: tuple[int, int | None] | None = None,
max_bytes: int | None = None,
) -> bytes
Download (a byte range of) an object. byte_range=(start, end_inclusive|None).
max_bytes caps the body: larger objects raise PAYLOAD_TOO_LARGE before any
byte is read (the control plane only ever reads small JSON sidecars).
stat(
bucket: str, name: str, *, generation: int | None = None
) -> ObjectMeta
Metadata-only objects.get; never downloads content.
list(
bucket: str,
*,
prefix: str = "",
delimiter: str | None = None,
page_token: str | None = None,
max_results: int = 1000,
) -> ObjectPage
List live objects under prefix in lexicographic name order.
delete(
bucket: str,
name: str,
*,
generation: int | None = None,
if_generation_match: int | None = None,
) -> None
Delete an object (or one generation). Retention-locked objects raise AccessDenied.
copy(
src_bucket: str,
src_name: str,
dst_bucket: str,
dst_name: str,
*,
source_generation: int | None = None,
if_generation_match: int | None = None,
) -> ObjectMeta
Server-side copy; hashes are recomputed by the service, bytes never leave it.
Finalisation pins source_generation to the verified generation and uses
if_generation_match=0 so bundle objects are write-once.
sign_get(
bucket: str,
name: str,
*,
expires_in: timedelta,
generation: int | None = None,
) -> SignedGet
Signed GET pinned to generation (default: the live one); Range requests work.
sign_put(
bucket: str,
name: str,
*,
expires_in: timedelta,
content_type: str | None = None,
max_bytes: int | None = None,
if_generation_match: int | None = 0,
) -> SignedPut
Signed PUT for exactly name (write-once unless if_generation_match=None).
post_policy(
bucket: str,
prefix: str,
*,
expires_in: timedelta,
max_bytes: int,
min_bytes: int = 0,
create_only: bool = False,
) -> PostPolicy
V4 POST policy allowing uploads of any key under prefix within the size range.
create_only uploads may not replace an existing object (whatever the signer's
own permissions).
Bases: StrictModel
Metadata-only view of one object generation (what objects.get returns).
crc32c and md5_hash are computed by the storage service, never by the uploader.
md5_hash is absent for composite objects (GCS semantics).
from_gcs_resource(
resource: Mapping[str, Any],
) -> ObjectMeta
Build from a GCS JSON-API object resource (generation/size are strings).
The GCS JSON-API shape (inverse of :meth:from_gcs_resource).
same_content(other: ObjectMeta) -> bool
Size and server-computed hashes agree (generation and location may differ).
Bases: StrictModel
One page of :meth:ObjectStore.list.
Bases: StrictModel
A generation-pinned signed GET URL plus the server-computed facts about its object.
The URL honours HTTP Range, so zarr zip readers can range-read one object. A reader
verifies size/crc32c after a full download (corruption check only — integrity is
enforced by the control plane).
Bases: StrictModel
A signed PUT URL for exactly one object name.
headers must be sent verbatim (they are part of the signature): typically
content-type, x-goog-if-generation-match: 0 (write-once) and
x-goog-content-length-range: 0,<max_bytes>.
Bases: StrictModel
A V4 POST policy: multipart form uploads of any key under prefix.
The uploader posts multipart/form-data to url with every entry of fields,
then key (which must start with prefix), then the file part last. The storage
service rejects bodies outside content_length_range and keys outside the prefix.
A create_only policy cannot replace an existing object (its signed conditions carry
x-goog-if-generation-match: 0), so the holder can neither overwrite the write-once
manifest nor swap an output after it was written.
Whether an upload of size bytes to key satisfies this policy.
Bases: StorageError
The principal may not perform this operation (e.g. listing the bootstrap bucket).
Bases: StrEnum
The five buckets of the data project (plan §8.6).
All bucket names for one environment (Buckets('local').runs == 'mapi-local-runs').
bucket_name(env: str, purpose: BucketPurpose | str) -> str
mapi-{env}-{purpose}, e.g. mapi-local-runs.
t/{tenant}/uploads/{upload_id} in the quarantine bucket.
{nonce} in the bootstrap bucket (the 256-bit nonce as 64 lowercase hex chars).
t/{tenant}/runs/{run}/a{n}/[tasks/{i}/] in the runs bucket (always ends in /).
run_checkpoint_prefix(
tenant_id: str,
run_id: str,
attempt: int,
task_index: int | None = None,
) -> str
{run_prefix}checkpoints/: where segmented sampling writes resumable state.
t/{tenant}/models/{model}/{version}/ in the bundles bucket.
t/{tenant}/datasets/{sha}/ in the data bucket (content-addressed).
Validate an object name: 1–1024 UTF-8 bytes, no control characters, no leading /,
no empty, . or .. segments (so keys can never escape a prefix).
Validate one key segment built from an ID ([A-Za-z0-9][A-Za-z0-9_.-]{0,127}).
Reject non-positive TTLs and TTLs above the V4 maximum.
CRC32C (Castagnoli) of data, continuing from value (for chunked hashing).
Pure Python; payloads above 64 KiB use google_crc32c when it is installed.
GCS wire form of a CRC32C value (base64 of 4 big-endian bytes).
GCS wire form of the MD5 of data.
REST DTOs¶
For the endpoint tables and the conventions, read HTTP API (v1). Also read the multi-tenant additions of v0.3.
REST DTOs for the HTTP API (/v1; plan §8.2): M1 (v0.2), M2 (v0.3) and the M3 (v0.4)
contracts.
Conventions shared by every endpoint (see docs/reference/http-api.md):
- Requests reject unknown fields (
extra="forbid"); responses ignore unknown fields when parsed by a client, so older clients keep working when the server adds fields. - Every resource carries
tenant_idandproject_id. M1 is single-tenant: both are :data:DEFAULT_TENANT_ID/ :data:DEFAULT_PROJECT_ID, but the identity is in every row and key so M2 can add row-level security without migrations churn. - Lists return :class:
Page(items+ opaquenext_cursor); errors are RFC 7807 :class:~mapi_spec.errors.Problemdocuments with stable codes. - Timestamps are timezone-aware RFC 3339 strings in UTC.
:data:M1_ENDPOINTS, :data:M2_ENDPOINTS (v0.3), :data:M3_ENDPOINTS (guided mode and
integrations, v0.4), :data:M35_ENDPOINTS (workflows, governance, explorers,
optimisation and About cards, v0.5), :data:M4_ENDPOINTS (tree datasets) and
:data:V011_ENDPOINTS (run tracking, v0.11) and :data:V014_ENDPOINTS (media, run
comparison, experiments and saved views, v0.14) and :data:V015_ENDPOINTS (resumable uploads and
image collections, v0.15) are the machine-readable endpoint tables;
:data:ENDPOINTS (= :data:ALL_ENDPOINTS) is all of them. The server's route table and the
reference docs are tested against them. :data:ROUTE_ACCESS gives every operation its
tenant/project scope and the permission it checks. From v0.3 every M1 resource is
project-scoped: the project comes from the path or the X-Mapi-Project header
(:func:mapi_spec.tenancy.resolve_project_scope).
M1_ENDPOINTS: Final[tuple[EndpointDecl, ...]] = (
EndpointDecl(
"GET",
"/healthz",
"health",
"Process liveness",
"Health",
200,
),
EndpointDecl(
"GET",
"/readyz",
"ready",
"Dependencies reachable (503 if not)",
"Health",
200,
),
EndpointDecl(
"GET",
"/v1/projects",
"list_projects",
"List projects",
"Page[Project]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/projects/{project_id}",
"get_project",
"Get a project",
"Project",
200,
errors=_NF,
),
EndpointDecl(
"GET",
"/v1/catalog",
"get_catalog",
"Full catalog",
"CatalogResponse",
200,
),
EndpointDecl(
"GET",
"/v1/catalog/models",
"list_catalog_models",
"Model plugins",
"Page[ModelCatalogEntry]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/catalog/models/{model_id}/{version}",
"get_catalog_model",
"One model plugin version (manifest, sanitized schemas)",
"ModelCatalogEntry",
200,
errors=_NF,
),
EndpointDecl(
"GET",
"/v1/catalog/steps",
"list_catalog_steps",
"Steps",
"Page[StepCatalogEntry]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/catalog/plots",
"list_catalog_plots",
"Plot recipes",
"Page[PlotCatalogEntry]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/catalog/plots/{plot_id}/{version}/example",
"get_catalog_plot_example",
"A plot recipe rendered on synthetic example data",
"PlotSpec",
200,
errors=_NF,
),
EndpointDecl(
"GET",
"/v1/schemas/{name}",
"get_schema",
"Exported JSON Schema (e.g. modelspec.json)",
None,
200,
errors=_NF,
),
EndpointDecl(
"POST",
"/v1/datasets",
"create_dataset",
"Create a dataset",
"Dataset",
201,
request="DatasetCreate",
errors=(*_BODY, _E.CONFLICT),
),
EndpointDecl(
"GET",
"/v1/datasets",
"list_datasets",
"List datasets",
"Page[Dataset]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/datasets/{dataset_id}",
"get_dataset",
"Get a dataset",
"Dataset",
200,
errors=_NF,
),
EndpointDecl(
"POST",
"/v1/datasets/{dataset_id}/uploads",
"create_upload",
"Signed PUT URL into quarantine",
"UploadSession",
201,
request="UploadCreate",
errors=(
*_BODY,
*_NF,
_E.PAYLOAD_TOO_LARGE,
_E.QUOTA_EXCEEDED,
),
),
EndpointDecl(
"POST",
"/v1/datasets/{dataset_id}/versions",
"create_dataset_version",
"Ingest an upload as a new version (starts an ingest run)",
"DatasetVersion",
202,
request="DatasetVersionCreate",
errors=(
*_BODY,
*_NF,
_E.CONFLICT,
_E.CAPABILITY_UNSUPPORTED,
_E.PAYLOAD_TOO_LARGE,
),
headers=("Idempotency-Key",),
),
EndpointDecl(
"GET",
"/v1/datasets/{dataset_id}/versions",
"list_dataset_versions",
"List versions",
"Page[DatasetVersion]",
200,
errors=_NF,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/datasets/{dataset_id}/versions/{version}",
"get_dataset_version",
"Schema, summary, profile and preview refs",
"DatasetVersion",
200,
errors=_NF,
),
EndpointDecl(
"GET",
"/v1/datasets/{dataset_id}/versions/{version}/artifacts/{name}",
"get_dataset_artifact",
"Signed URL for data.zarr.zip / preview / profile / schema",
"ArtifactLink",
200,
errors=_NF,
),
EndpointDecl(
"POST",
"/v1/specs:validate",
"validate_spec",
"Synchronous static validation (issues at JSON pointers, per-stage schemas)",
"ValidateResponse",
200,
request="ValidateRequest",
errors=(
_E.SPEC_PARSE_ERROR,
_E.PAYLOAD_TOO_LARGE,
_E.SCHEMA_INVALID,
),
),
EndpointDecl(
"POST",
"/v1/specs",
"create_spec",
"Create a spec with revision 1",
"SpecDoc",
201,
request="SpecCreate",
errors=(
_E.SPEC_PARSE_ERROR,
_E.PAYLOAD_TOO_LARGE,
_E.SCHEMA_INVALID,
_E.CONFLICT,
),
),
EndpointDecl(
"GET",
"/v1/specs",
"list_specs",
"List specs",
"Page[SpecDoc]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/specs/{spec_id}",
"get_spec",
"Get a spec",
"SpecDoc",
200,
errors=_NF,
),
EndpointDecl(
"POST",
"/v1/specs/{spec_id}/revisions",
"create_spec_revision",
"Add an immutable revision (409 if parent_revision is stale)",
"SpecRevision",
201,
request="SpecRevisionCreate",
errors=(
*_NF,
_E.SPEC_PARSE_ERROR,
_E.PAYLOAD_TOO_LARGE,
_E.SCHEMA_INVALID,
_E.CONFLICT,
),
),
EndpointDecl(
"GET",
"/v1/specs/{spec_id}/revisions",
"list_spec_revisions",
"List revisions",
"Page[SpecRevision]",
200,
errors=_NF,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/specs/{spec_id}/revisions/{revision}",
"get_spec_revision",
"Get one revision",
"SpecRevision",
200,
errors=_NF,
),
EndpointDecl(
"POST",
"/v1/specs/{spec_id}/revisions/{revision}:dry-run",
"dry_run_spec",
"Sandboxed dry-run (a 'validate' run)",
"Run",
202,
errors=(*_NF, _E.QUOTA_EXCEEDED),
headers=("Idempotency-Key",),
),
EndpointDecl(
"POST",
"/v1/runs",
"create_run",
"Submit a run (200 on idempotent replay)",
"Run",
201,
request="RunCreate",
errors=(
*_BODY,
*_NF,
_E.IDEMPOTENCY_CONFLICT,
_E.QUOTA_EXCEEDED,
_E.MODE_NOT_SUPPORTED,
_E.LEAKAGE_RISK,
_E.UNSEEN_LEVELS,
_E.COORD_NOT_ALLOWED,
_E.HORIZON_EXCEEDS_FIT,
_E.MISSING_FUTURE_COVARIATE,
_E.RECURSIVE_UNSUPPORTED,
_E.ACTUALS_NOT_SUPPORTED,
_E.ACTUALS_INVALID,
),
headers=("Idempotency-Key",),
),
EndpointDecl(
"GET",
"/v1/runs",
"list_runs",
"List runs (filters: kind, state, spec, parent, experiment, name, tags, metric/param search)",
"Page[Run]",
200,
errors=_BODY,
paginated=True,
query=(
*_PAGE,
"kind",
"state",
"spec_id",
"parent_run_id",
"name",
"tag",
"filter",
"order_by",
"column",
"experiment_id",
),
),
EndpointDecl(
"GET",
"/v1/runs/{run_id}",
"get_run",
"Get a run",
"Run",
200,
errors=_NF,
),
EndpointDecl(
"GET",
"/v1/runs/{run_id}/events",
"run_events",
"SSE stream (Accept: text/event-stream) or JSON page of persisted events",
"Page[RunEventView]",
200,
errors=_NF,
headers=("Last-Event-ID",),
paginated=True,
streaming=True,
query=("after", "limit"),
),
EndpointDecl(
"GET",
"/v1/runs/{run_id}/logs",
"run_logs",
"Captured log lines (untrusted text)",
"Page[LogLine]",
200,
errors=_NF,
paginated=True,
query=(
*_PAGE,
"attempt",
"stream",
"level",
"include_user",
),
),
EndpointDecl(
"GET",
"/v1/runs/{run_id}/plots",
"list_run_plots",
"Sanitized plots a finished run rendered (score/evaluate fans, counterfactuals)",
"Page[PlotSpec]",
200,
errors=(*_NF, _E.INTEGRITY_FAILURE),
paginated=True,
query=_PAGE,
),
EndpointDecl(
"POST",
"/v1/runs/{run_id}:cancel",
"cancel_run",
"Request cancellation (idempotent)",
"Run",
202,
errors=(*_NF, _E.CONFLICT),
),
EndpointDecl(
"POST",
"/v1/runs/{run_id}:resume",
"resume_run",
"Resume a PARTIAL run",
"Run",
202,
errors=(*_NF, _E.CONFLICT),
),
EndpointDecl(
"POST",
"/v1/runs/{run_id}:retry",
"retry_run",
"New run from a failed one",
"Run",
201,
request="RunRetry",
errors=(*_BODY, *_NF, _E.CONFLICT),
headers=("Idempotency-Key",),
),
EndpointDecl(
"GET",
"/v1/models",
"list_models",
"List models",
"Page[Model]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/models/{model_id}",
"get_model",
"Get a model",
"Model",
200,
errors=_NF,
),
EndpointDecl(
"GET",
"/v1/models/{model_id}/versions",
"list_model_versions",
"List versions",
"Page[ModelVersion]",
200,
errors=_NF,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/models/{model_id}/versions/{version}",
"get_model_version",
"Signed manifest, diagnostics, scorecard, lineage, metrics",
"ModelVersion",
200,
errors=_NF,
),
EndpointDecl(
"GET",
"/v1/models/{model_id}/versions/{version}/artifacts/{name}",
"get_model_artifact",
"Signed URL for a bundle object or ui/ view (name may contain /)",
"ArtifactLink",
200,
errors=(*_NF, _E.INTEGRITY_FAILURE),
),
EndpointDecl(
"POST",
"/v1/models/{model_id}/versions/{version}:score",
"score_model",
"Score with replayed preprocessing (Prefer: wait=N → 200 if done, else 202)",
"ScoreCallResult",
200,
request="ScoreCall",
errors=(
*_BODY,
*_NF,
_E.MODE_NOT_SUPPORTED,
_E.UNSEEN_LEVELS,
_E.COORD_NOT_ALLOWED,
_E.HORIZON_EXCEEDS_FIT,
_E.MISSING_FUTURE_COVARIATE,
_E.NON_CONTIGUOUS_EXTENSION,
_E.CAPABILITY_UNSUPPORTED,
_E.RECURSIVE_UNSUPPORTED,
_E.QUOTA_EXCEEDED,
_E.ACTUALS_NOT_SUPPORTED,
_E.ACTUALS_INVALID,
_E.PAYLOAD_TOO_LARGE,
),
headers=("Prefer", "Idempotency-Key"),
),
EndpointDecl(
"GET",
"/v1/models/{model_id}/versions/{version}/inputs",
"get_model_inputs",
"What a score call can send: the input contract, the fitted coordinates and the input table",
"ModelInputSchema",
200,
errors=(*_NF, _E.CAPABILITY_UNSUPPORTED),
query=("periods",),
),
EndpointDecl(
"GET",
"/v1/models/{model_id}/versions/{version}/plots",
"list_plots",
"Plots of a model version",
"Page[PlotListItem]",
200,
errors=_NF,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/models/{model_id}/versions/{version}/plots/{plot_id}",
"get_plot",
"A sanitized PlotSpec (render it in the plot sandbox origin)",
"PlotSpec",
200,
errors=_NF,
),
EndpointDecl(
"POST",
"/v1/plots:validate",
"validate_plot",
"Run the plot sanitizer",
"PlotValidateResponse",
200,
request="PlotSpec",
errors=_BODY,
),
)
The M1 endpoint table. request/response name DTOs in this module.
M2_ENDPOINTS: Final[tuple[EndpointDecl, ...]] = (
EndpointDecl(
"GET",
"/v1/me",
"get_me",
"The caller, their tenants, projects and roles",
"Me",
200,
errors=(_E.UNAUTHENTICATED,),
),
EndpointDecl(
"GET",
"/v1/tenants",
"list_tenants",
"Tenants the caller belongs to",
"Page[Tenant]",
200,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/tenants/{tenant_id}",
"get_tenant",
"Get a tenant",
"Tenant",
200,
errors=_NF,
),
EndpointDecl(
"PATCH",
"/v1/tenants/{tenant_id}",
"update_tenant",
"Rename a tenant (tenant admin)",
"Tenant",
200,
request="TenantUpdate",
errors=(*_BODY, *_NF, *_FB),
),
EndpointDecl(
"GET",
"/v1/tenants/{tenant_id}/projects",
"list_tenant_projects",
"Projects of a tenant visible to the caller",
"Page[Project]",
200,
errors=_NF,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"POST",
"/v1/tenants/{tenant_id}/projects",
"create_project",
"Create a project (tenant admin)",
"Project",
201,
request="ProjectCreate",
errors=(*_BODY, *_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"PATCH",
"/v1/tenants/{tenant_id}/projects/{project_id}",
"update_project",
"Rename or describe a project (project admin)",
"Project",
200,
request="ProjectUpdate",
errors=(*_BODY, *_NF, *_FB),
),
EndpointDecl(
"DELETE",
"/v1/tenants/{tenant_id}/projects/{project_id}",
"delete_project",
"Delete an empty project (tenant admin; 409 while it holds data)",
None,
204,
errors=(*_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"GET",
"/v1/tenants/{tenant_id}/members",
"list_tenant_members",
"Tenant-wide members",
"Page[Membership]",
200,
errors=_NF,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"POST",
"/v1/tenants/{tenant_id}/members",
"add_tenant_member",
"Grant a tenant-wide role (tenant admin)",
"Membership",
201,
request="MembershipCreate",
errors=(*_BODY, *_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"PATCH",
"/v1/tenants/{tenant_id}/members/{principal_id}",
"update_tenant_member",
"Change a tenant-wide role (tenant admin)",
"Membership",
200,
request="MembershipUpdate",
errors=(*_BODY, *_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"DELETE",
"/v1/tenants/{tenant_id}/members/{principal_id}",
"remove_tenant_member",
"Remove a tenant-wide role (409 for the last admin)",
None,
204,
errors=(*_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"GET",
"/v1/tenants/{tenant_id}/projects/{project_id}/members",
"list_members",
"Project members (including tenant-wide grants)",
"Page[Membership]",
200,
errors=_NF,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"POST",
"/v1/tenants/{tenant_id}/projects/{project_id}/members",
"add_member",
"Grant a project role (project admin)",
"Membership",
201,
request="MembershipCreate",
errors=(*_BODY, *_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"PATCH",
"/v1/tenants/{tenant_id}/projects/{project_id}/members/{principal_id}",
"update_member",
"Change a project role (project admin)",
"Membership",
200,
request="MembershipUpdate",
errors=(*_BODY, *_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"DELETE",
"/v1/tenants/{tenant_id}/projects/{project_id}/members/{principal_id}",
"remove_member",
"Remove a project role (409 for the last admin)",
None,
204,
errors=(*_NF, *_FB, _E.CONFLICT),
),
EndpointDecl(
"GET",
"/v1/tenants/{tenant_id}/usage",
"get_tenant_usage",
"Cost and quota of the whole tenant for a period (tenant admin)",
"UsageReport",
200,
errors=(*_NF, *_FB),
query=("period", "start", "group_by"),
),
EndpointDecl(
"GET",
"/v1/tenants/{tenant_id}/audit",
"list_tenant_audit",
"The tenant's audit log (tenant admin)",
"Page[AuditEvent]",
200,
errors=(*_NF, *_FB),
paginated=True,
query=(
*_PAGE,
"project_id",
"actor_id",
"action",
"outcome",
"since",
"until",
),
),
EndpointDecl(
"GET",
"/v1/tokens",
"list_tokens",
"The caller's tokens in the project (all=true: everyone's, project admin)",
"Page[ApiToken]",
200,
paginated=True,
query=(*_PAGE, "all", "status"),
headers=_PH,
),
EndpointDecl(
"POST",
"/v1/tokens",
"create_token",
"Mint a token (the secret is shown once)",
"TokenCreated",
201,
request="TokenCreate",
errors=(*_BODY, *_FB),
headers=_PH,
),
EndpointDecl(
"GET",
"/v1/tokens/{token_id}",
"get_token",
"Get a token (never its secret)",
"ApiToken",
200,
errors=_NF,
headers=_PH,
),
EndpointDecl(
"DELETE",
"/v1/tokens/{token_id}",
"revoke_token",
"Revoke a token (idempotent)",
None,
204,
errors=(*_NF, *_FB),
headers=_PH,
),
EndpointDecl(
"GET",
"/v1/usage",
"get_usage",
"Cost and quota of the project for a period",
"UsageReport",
200,
query=("period", "start", "group_by"),
headers=_PH,
),
EndpointDecl(
"GET",
"/v1/admin/tenants",
"admin_list_tenants",
"All tenants (filters: status, tier)",
"Page[Tenant]",
200,
errors=_FB,
paginated=True,
query=(*_PAGE, "status", "trust_tier"),
),
EndpointDecl(
"POST",
"/v1/admin/tenants",
"admin_create_tenant",
"Create a tenant",
"Tenant",
201,
request="TenantCreate",
errors=(*_BODY, *_FB, _E.CONFLICT),
),
EndpointDecl(
"GET",
"/v1/admin/tenants/{tenant_id}",
"admin_get_tenant",
"Get any tenant",
"Tenant",
200,
errors=(*_NF, *_FB),
),
EndpointDecl(
"PATCH",
"/v1/admin/tenants/{tenant_id}",
"admin_update_tenant",
"Change a tenant's tier, status or residency",
"Tenant",
200,
request="TenantAdminUpdate",
errors=(*_BODY, *_NF, *_FB),
),
EndpointDecl(
"GET",
"/v1/admin/quotas",
"admin_list_quotas",
"Quota policies (filters: scope, subject)",
"Page[QuotaPolicy]",
200,
errors=_FB,
paginated=True,
query=(*_PAGE, "scope", "subject_id"),
),
EndpointDecl(
"PUT",
"/v1/admin/quotas",
"admin_put_quota",
"Create or replace one quota policy",
"QuotaPolicy",
200,
request="QuotaPolicyUpsert",
errors=(*_BODY, *_FB),
),
EndpointDecl(
"DELETE",
"/v1/admin/quotas/{scope}/{subject_id}",
"admin_delete_quota",
"Remove a policy (the tier default applies again)",
None,
204,
errors=(*_NF, *_FB),
),
EndpointDecl(
"GET",
"/v1/admin/kill-switches",
"admin_get_kill_switches",
"Current kill switches",
"KillSwitches",
200,
errors=_FB,
),
EndpointDecl(
"PUT",
"/v1/admin/kill-switches",
"admin_put_kill_switches",
"Flip kill switches (audited; reason required)",
"KillSwitches",
200,
request="KillSwitchUpdate",
errors=(*_BODY, *_FB),
),
EndpointDecl(
"GET",
"/v1/admin/executors",
"admin_list_executors",
"Executor health, caps and conformance",
"Page[ExecutorHealth]",
200,
errors=_FB,
paginated=True,
query=_PAGE,
),
EndpointDecl(
"GET",
"/v1/admin/executors/{name}",
"admin_get_executor",
"One executor",
"ExecutorHealth",
200,
errors=(*_NF, *_FB),
),
EndpointDecl(
"GET",
"/v1/admin/executors/{name}/evidence",
"admin_get_executor_evidence",
"The conformance and security-suite reports recorded for an executor",
"ExecutorEvidence",
200,
errors=(*_NF, *_FB),
),
EndpointDecl(
"PUT",
"/v1/admin/executors/{name}/evidence",
"admin_put_executor_evidence",
"Record suite results (audited); shared tiers need both, passing, for the current caps",
"ExecutorEvidence",
200,
request="ExecutorEvidenceUpdate",
errors=(*_BODY, *_FB),
),
EndpointDecl(
"GET",
"/v1/admin/usage",
"admin_get_usage",
"Cost by tenant for a period and current spend against the global cap",
"AdminUsageReport",
200,
errors=_FB,
query=("period", "start"),
),
EndpointDecl(
"PUT",
"/v1/admin/executors/{name}",
"admin_put_executor",
"Enable or disable an executor (audited; reason required)",
"ExecutorHealth",
200,
request="ExecutorUpdate",
errors=(*_BODY, *_NF, *_FB),
),
EndpointDecl(
"GET",
"/v1/admin/audit",
"admin_list_audit",
"The platform audit log",
"Page[AuditEvent]",
200,
errors=_FB,
paginated=True,
query=(
*_PAGE,
"tenant_id",
"project_id",
"actor_id",
"action",
"outcome",
"since",
"until",
),
),
EndpointDecl(
"GET",
"/auth/login",
"auth_login",
"Start OIDC sign-in (302 to the identity provider; Authorization Code + PKCE)",
None,
302,
query=("return_to",),
),
EndpointDecl(
"GET",
"/auth/callback",
"auth_callback",
"Finish sign-in: verify state, exchange the code, set the session cookie, 302 back",
None,
302,
errors=(_E.UNAUTHENTICATED,),
query=("code", "state", "error"),
),
EndpointDecl(
"POST",
"/auth/logout",
"auth_logout",
"End the session and clear the cookie",
None,
204,
),
EndpointDecl(
"GET",
"/auth/session",
"auth_session",
"Is the browser signed in, as whom",
"SessionInfo",
200,
),
EndpointDecl(
"POST",
"/internal/events",
"ingest_event",
"Pub/Sub push of run events (OIDC-authenticated service account only)",
None,
200,
request="PubSubPushEnvelope",
errors=(_E.UNAUTHENTICATED, _E.SCHEMA_INVALID),
),
)
The M2 endpoint table (additive to :data:M1_ENDPOINTS).
V03_ENDPOINTS: Final[tuple[EndpointDecl, ...]] = (
M1_ENDPOINTS + M2_ENDPOINTS
)
Every endpoint of the v0.3 API (M1 + M2).
ENDPOINTS: Final[tuple[EndpointDecl, ...]] = (
V04_ENDPOINTS
+ M35_ENDPOINTS
+ M4_ENDPOINTS
+ V011_ENDPOINTS
+ V014_ENDPOINTS
+ V015_ENDPOINTS
)
Every endpoint of the API (M1 + M2 + M3 + M3.5 + M4 + v0.11 + v0.14 + v0.15); the server serves every row.
ROUTE_ACCESS: Final[dict[str, RouteAccess]] = {
"health": _PUBLIC,
"ready": _PUBLIC,
"get_schema": _PUBLIC,
"list_projects": _AUTH,
"get_project": _p(_RK.PROJECT, _AC.READ),
"get_catalog": _AUTH,
"list_catalog_models": _AUTH,
"get_catalog_model": _AUTH,
"list_catalog_steps": _AUTH,
"list_catalog_plots": _AUTH,
"get_catalog_plot_example": _AUTH,
"create_dataset": _p(_RK.DATASET, _AC.CREATE),
"list_datasets": _p(_RK.DATASET, _AC.READ),
"get_dataset": _p(_RK.DATASET, _AC.READ),
"create_upload": _p(_RK.DATASET, _AC.UPDATE),
"create_dataset_version": _p(_RK.DATASET, _AC.UPDATE),
"list_dataset_versions": _p(_RK.DATASET, _AC.READ),
"get_dataset_version": _p(_RK.DATASET, _AC.READ),
"get_dataset_artifact": _p(_RK.DATASET, _AC.READ),
"validate_spec": _p(_RK.SPEC, _AC.READ),
"create_spec": _p(_RK.SPEC, _AC.CREATE),
"list_specs": _p(_RK.SPEC, _AC.READ),
"get_spec": _p(_RK.SPEC, _AC.READ),
"create_spec_revision": _p(_RK.SPEC, _AC.UPDATE),
"list_spec_revisions": _p(_RK.SPEC, _AC.READ),
"get_spec_revision": _p(_RK.SPEC, _AC.READ),
"dry_run_spec": _p(
_RK.RUN, _AC.CREATE, (_RK.SPEC, _AC.READ)
),
"create_run": _p(
_RK.RUN, _AC.CREATE, (_RK.MODEL, _AC.RUN)
),
"list_runs": _p(_RK.RUN, _AC.READ),
"get_run": _p(_RK.RUN, _AC.READ),
"run_events": _p(_RK.RUN, _AC.READ),
"run_logs": _p(_RK.RUN, _AC.READ),
"list_run_plots": _p(_RK.RUN, _AC.READ),
"cancel_run": _p(_RK.RUN, _AC.CANCEL),
"resume_run": _p(_RK.RUN, _AC.CREATE),
"retry_run": _p(_RK.RUN, _AC.CREATE),
"list_models": _p(_RK.MODEL, _AC.READ),
"get_model": _p(_RK.MODEL, _AC.READ),
"list_model_versions": _p(_RK.MODEL, _AC.READ),
"get_model_version": _p(_RK.MODEL, _AC.READ),
"get_model_artifact": _p(_RK.MODEL, _AC.READ),
"score_model": RouteAccess(
RouteScope.PROJECT,
_RK.MODEL,
_AC.RUN,
name_param="model_id",
),
"get_model_inputs": _p(_RK.MODEL, _AC.READ),
"list_plots": _p(_RK.PLOT, _AC.READ),
"get_plot": _p(_RK.PLOT, _AC.READ),
"validate_plot": _p(_RK.PLOT, _AC.READ),
"get_me": _AUTH,
"list_tenants": _AUTH,
"get_tenant": _t(_RK.TENANT, _AC.READ),
"update_tenant": _t(_RK.TENANT, _AC.UPDATE),
"list_tenant_projects": _t(),
"create_project": _t(_RK.PROJECT, _AC.CREATE),
"update_project": _p(_RK.PROJECT, _AC.UPDATE),
"delete_project": _t(_RK.PROJECT, _AC.DELETE),
"list_tenant_members": _t(_RK.TENANT_MEMBER, _AC.READ),
"add_tenant_member": _t(_RK.TENANT_MEMBER, _AC.CREATE),
"update_tenant_member": _t(
_RK.TENANT_MEMBER, _AC.UPDATE
),
"remove_tenant_member": _t(
_RK.TENANT_MEMBER, _AC.DELETE
),
"list_members": _p(_RK.MEMBER, _AC.READ),
"add_member": _p(_RK.MEMBER, _AC.CREATE),
"update_member": _p(_RK.MEMBER, _AC.UPDATE),
"remove_member": _p(_RK.MEMBER, _AC.DELETE),
"get_tenant_usage": _t(_RK.BILLING, _AC.READ),
"list_tenant_audit": _t(_RK.AUDIT, _AC.READ),
"list_tokens": _p(_RK.TOKEN, _AC.READ),
"create_token": _p(_RK.TOKEN, _AC.CREATE),
"get_token": _p(_RK.TOKEN, _AC.READ),
"revoke_token": _p(_RK.TOKEN, _AC.DELETE),
"get_usage": _p(_RK.USAGE, _AC.READ),
"admin_list_tenants": _op(_RK.TENANT, _AC.READ),
"admin_create_tenant": _op(_RK.TENANT, _AC.CREATE),
"admin_get_tenant": _op(_RK.TENANT, _AC.READ),
"admin_update_tenant": _op(_RK.TENANT, _AC.MANAGE),
"admin_list_quotas": _op(_RK.QUOTA, _AC.READ),
"admin_put_quota": _op(_RK.QUOTA, _AC.UPDATE),
"admin_delete_quota": _op(_RK.QUOTA, _AC.DELETE),
"admin_get_kill_switches": _op(
_RK.KILL_SWITCH, _AC.READ
),
"admin_put_kill_switches": _op(
_RK.KILL_SWITCH, _AC.UPDATE
),
"admin_list_executors": _op(_RK.EXECUTOR, _AC.READ),
"admin_get_executor": _op(_RK.EXECUTOR, _AC.READ),
"admin_get_executor_evidence": _op(
_RK.EXECUTOR, _AC.READ
),
"admin_put_executor_evidence": _op(
_RK.EXECUTOR, _AC.UPDATE
),
"admin_put_executor": _op(_RK.EXECUTOR, _AC.UPDATE),
"admin_list_audit": _op(_RK.AUDIT, _AC.READ),
"admin_get_usage": _op(_RK.BILLING, _AC.READ),
"auth_login": _PUBLIC,
"auth_callback": _PUBLIC,
"auth_logout": _PUBLIC,
"auth_session": _PUBLIC,
"ingest_event": RouteAccess(
RouteScope.INTERNAL, _RK.EVENT, _AC.CREATE
),
"list_guided_templates": _AUTH,
"get_guided_template": _AUTH,
"detect_roles": _p(_RK.DATASET, _AC.READ),
"create_guided_draft": _p(_RK.SPEC, _AC.READ),
"compile_guided_request": _p(_RK.MODEL, _AC.READ),
"create_prior_translation": _p(
_RK.RUN, _AC.CREATE, (_RK.SPEC, _AC.READ)
),
"get_prior_translation": _p(_RK.RUN, _AC.READ),
"list_endpoints": _p(_RK.ENDPOINT, _AC.READ),
"get_endpoint": _alias(_AC.READ),
"promote_endpoint": _alias(
_AC.PROMOTE, (_RK.MODEL, _AC.READ)
),
"delete_endpoint": _alias(_AC.DELETE),
"list_endpoint_history": _alias(_AC.READ),
"score_endpoint": _alias(_AC.RUN),
"get_endpoint_schema": _alias(_AC.READ),
"validate_endpoint_request": _alias(_AC.READ),
"rollback_endpoint": _alias(_AC.PROMOTE),
"list_endpoint_snippets": _alias(_AC.READ),
"list_webhooks": _p(_RK.WEBHOOK, _AC.READ),
"create_webhook": _p(_RK.WEBHOOK, _AC.CREATE),
"get_webhook": _p(_RK.WEBHOOK, _AC.READ),
"update_webhook": _p(_RK.WEBHOOK, _AC.UPDATE),
"delete_webhook": _p(_RK.WEBHOOK, _AC.DELETE),
"rotate_webhook_secret": _p(_RK.WEBHOOK, _AC.UPDATE),
"list_webhook_deliveries": _p(_RK.WEBHOOK, _AC.READ),
"replay_webhook_delivery": _p(_RK.WEBHOOK, _AC.UPDATE),
"create_saved_plot": _p(_RK.PLOT, _AC.CREATE),
"list_saved_plots": _p(_RK.PLOT, _AC.READ),
"get_saved_plot": _p(_RK.PLOT, _AC.READ),
"delete_saved_plot": _p(_RK.PLOT, _AC.DELETE),
"render_plot": _p(_RK.PLOT, _AC.RUN),
"get_plot_render": _p(_RK.PLOT, _AC.READ),
"create_plot_embed": _p(
_RK.EMBED, _AC.CREATE, (_RK.PLOT, _AC.READ)
),
"list_plot_embeds": _p(_RK.EMBED, _AC.READ),
"revoke_plot_embed": _p(_RK.EMBED, _AC.DELETE),
"get_embed": _PUBLIC,
"signup": _PUBLIC,
"verify_email": _PUBLIC,
"get_tenant_verification": _t(_RK.TENANT, _AC.READ),
"admin_put_tenant_verification": _op(
_RK.TENANT, _AC.MANAGE
),
"request_tenant_deletion": _t(
_RK.TENANT_DELETION, _AC.CREATE
),
"get_tenant_deletion": _t(
_RK.TENANT_DELETION, _AC.READ
),
"cancel_tenant_deletion": _t(
_RK.TENANT_DELETION, _AC.CANCEL
),
"create_tenant_export": _t(
_RK.TENANT_EXPORT, _AC.CREATE
),
"list_tenant_exports": _t(_RK.TENANT_EXPORT, _AC.READ),
"get_tenant_export": _t(_RK.TENANT_EXPORT, _AC.READ),
"admin_list_signups": _op(_RK.TENANT, _AC.READ),
"admin_review_signup": _op(_RK.TENANT, _AC.MANAGE),
"admin_retry_tenant_deletion": _op(
_RK.TENANT_DELETION, _AC.MANAGE
),
**_M35_ACCESS,
**_M4_ACCESS,
**_V011_ACCESS,
**_V014_ACCESS,
**_V015_ACCESS,
}
Tenant/project scoping and the permission checked for every operation in
:data:ALL_ENDPOINTS (keyed by operation_id; M3.5 rows from :data:_M35_ACCESS).
Acting on other principals' tokens (list_tokens?all=true, revoking someone else's
token) additionally needs manage on token. A create_run for a model version
(score, evaluate, derive, render) additionally needs run on model. A
promote_endpoint with override=true additionally needs manage on endpoint,
which no token scope grants (a signed-in admin decides, audited with the reason).
Bases: ResponseModel, Generic[T]
One page of a list endpoint. Pass next_cursor back as ?cursor= for the next
page; None means this is the last page. ?limit= defaults to 50 (max 200).
Bases: ResponseModel
Bases: RequestModel
Create an (empty) dataset; data arrives as versions.
Bases: RequestModel
Ask for a signed PUT URL into the quarantine bucket for one file.
Bases: ResponseModel
Where to PUT the file. upload.headers must be sent verbatim; the URL expires.
Bases: RequestModel
How an upload becomes a labelled dataset (Meridian CoordToColumns style).
Tabular formats need a mapping; zarr_zip/netcdf are self-describing and may omit
dims/vars (with groups, they rename/select per node by name). A raw hdf5
file (M4) needs hdf5 (:class:Hdf5Mapping) instead of dims/vars. Parsing
happens only inside a sandboxed ingest run.
With the mff preset the long table is first pivoted to one column
per VariableName (plus Period and the key columns); vars and wide then
pick and rename those variables (column = a VariableName value) and dims may
refine the dims the preset derives (column = an MFF column). Without vars and
wide every variable is kept under its own name.
effective_mff: MffOptions | None
The MFF options the ingest run uses (defaults for the mff preset).
Bases: RequestModel
A dim of the dataset, read from one column of a tabular upload.
Bases: RequestModel
A data variable read from one column (dtype is inferred during ingest).
Bases: RequestModel
Turn finished uploads into a new version via a sandboxed ingest run (202).
Exactly one of: upload_id + mapping (one file), assembly (M4: several
files combined per node into one version, see :class:IngestAssembly), or
collection (v0.15: an image collection from a manifest and image parts, see
:class:~mapi_spec.collection.CollectionSource).
Every upload this version consumes (source order; a collection: manifest, parts).
Bases: ResponseModel
An immutable, content-addressed version (data.zarr.zip + schema + profile).
Bases: StrEnum
Bases: RequestModel
Create a spec document with its first revision (YAML 1.2 or JSON text).
Bases: ResponseModel
A named spec and its latest revision.
Bases: RequestModel
Add an immutable revision. parent_revision guards against lost updates (409).
Bases: ResponseModel
One immutable, canonically hashed revision. issues holds its static validation.
Bases: RequestModel
Synchronous static validation of a document (exactly one of yaml/document).
dataset overrides the dataset version whose stored schema is the raw input schema
(default: the spec's own data.dataset reference, resolved now).
Bases: ResponseModel
Every problem at once, located by JSON pointers, plus per-stage schemas.
Bases: ResponseModel
The data schema after one pipeline step (raw = after binding, before any step).
Bases: RequestModel
Submit a run. Send an Idempotency-Key header: a retry with the same key and body
returns the original run (200); the same key with a different body is 409
IDEMPOTENCY_CONFLICT. Keys are scoped to the project and kept for 24 h.
fit/validate(sandboxed dry-run):spec_id+revision;score:model_version_id+request;evaluate/derive/render:model_version_id(+params).
Bases: RequestModel
POST /runs/{id}:retry: a new run from a failed one, optionally one class up.
Bases: ResponseModel
A unit of work and its lifecycle (see the state machine in mapi_spec.run).
Bases: ResponseModel
Latest coalesced progress (one upserted row per run).
Bases: ResponseModel
A persisted run event (id is the SSE event id, increasing per run).
Bases: ResponseModel
One captured line. line is untrusted text: render it as text only.
Bases: StrEnum
event: names on GET /v1/runs/{id}/events (text/event-stream).
snapshot (the full :class:Run, no id) is sent first on every connection; run-event
types carry id: = :attr:RunEventView.id; tracking (a :class:TrackingFrame, no
id, v0.11) tells that metric series, params or tags changed; end is sent once the run
is terminal, then the server closes the stream. The metrics, params and tags
run events are not stored as events, so they have no frame name of their own.
Bases: ResponseModel
A registry entry grouping the versions fitted from one spec name.
Bases: ResponseModel
A verified bundle: signed manifest, diagnostics, scorecard, lineage and metrics.
Bases: ResponseModel
Sanity scorecard: convergence, holdout error and plausibility checks rolled up.
Bases: ResponseModel
Bases: ResponseModel
Everything a model version came from (plan §8.6).
Bases: RequestModel
Body of POST /v1/models/{model_id}/versions/{version}:score.
Send Prefer: wait=N (N ≤ 60) to wait for the result: the response is 200 with
outputs if the run finished in time, else 202 with the run (poll it or follow its SSE).
inline carries data/scenario typed in instead of named as datasets; the
request then names neither.
Bases: ResponseModel
done with output links (200), or the still-running run (202). summary is the
requested summary (mean and HDI by default) inline, once the run succeeded.
Bases: ResponseModel
A short-lived, generation-pinned signed GET URL for one artifact (HTTP Range works).
Bases: ResponseModel
A stored object exposed through …/artifacts/{name} (fetch it as an ArtifactLink).
Bases: ResponseModel
A plot available for a model version; fetch the sanitized :class:PlotSpec at href.
Bases: ResponseModel
Result of POST /v1/plots:validate (the server-side plot sanitizer).
Bases: ResponseModel
The static catalog the control plane validates against (no plugin code involved).
Bases: ResponseModel
/healthz (process liveness) and /readyz (dependencies) body.
Bases: ResponseModel
Bases: ResponseModel
GET /v1/me: who the caller is and where they may act.
For a token, tenants holds exactly the token's project and token the token
itself (its scopes narrow every role shown here).
Bases: ResponseModel
A tenant the caller belongs to; role is the tenant-wide role (None when the
caller only has project memberships).
Bases: ResponseModel
A project the caller can act in, with the caller's effective role there.
Bases: RequestModel
POST /v1/admin/tenants (operators). initial_admin_email is invited as tenant
admin; the invitation is claimed at first login with that verified e-mail.
Bases: RequestModel
PATCH /v1/tenants/{tenant_id} (tenant admins): the display name only.
Bases: RequestModel
PATCH /v1/admin/tenants/{tenant_id} (operators). Changing data_residency only
affects where new runs are placed; moving stored data is a separate procedure.
Bases: RequestModel
POST /v1/tenants/{tenant_id}/projects (tenant admins).
Bases: RequestModel
PATCH /v1/tenants/{tenant_id}/projects/{project_id} (project admins).
Bases: RequestModel
Add a member (tenant-wide or to one project, by path). Give a known principal_id
or an email to invite (claimed at first login with that verified e-mail).
Bases: RequestModel
Change a member's role (the last admin of a tenant or project cannot be demoted)
and, from M3.5, their capabilities (reviewer).
Bases: ResponseModel
GET /v1/usage (one project) and GET /v1/tenants/{tenant_id}/usage (whole
tenant): ledger totals for one UTC period and the quota limits in force now.
Query: period=day|month (default month), start (an RFC 3339 instant inside
the period; default: now), group_by.
EndpointDecl(
method: Literal[
"GET", "POST", "PUT", "PATCH", "DELETE"
],
path: str,
operation_id: str,
summary: str,
response: str | None,
success: int,
request: str | None = None,
errors: tuple[ErrorCode, ...] = (),
headers: tuple[str, ...] = (),
paginated: bool = False,
streaming: bool = False,
query: tuple[str, ...] = (),
)
One row of an endpoint table (used to test the server routes and the docs).
response names a DTO (Page[X] for lists); None means no JSON body
(204, 302) or a server-defined body (/internal/events).
Bases: StrEnum
What a route needs before its permission check (see :data:ROUTE_ACCESS).
Any principal; results are filtered to what the caller can see.
RouteAccess(
scope: RouteScope,
resource: ResourceKind | None = None,
action: Action | None = None,
also: tuple[tuple[ResourceKind, Action], ...] = (),
name_param: str | None = None,
)
How one operation is scoped and authorized.
resource/action name the :data:~mapi_spec.tenancy.PERMISSIONS entry checked
with :meth:~mapi_spec.tenancy.AuthContext.allows; None means membership in the
scope suffices (lists are filtered by visibility). Handlers add the secondary checks in
also when they apply (e.g. a score run also needs run on the model).
name_param is the path parameter that names the resource, passed as
resource_name so targeted scopes (score:<model id>) can match it.
route_access(operation_id: str) -> RouteAccess
The :class:RouteAccess of an operation (KeyError for unknown operations).
encode_sse(
event: SseEventName | str,
data: BaseModel | dict[str, Any],
*,
id: int | str | None = None,
retry_ms: int | None = None,
) -> str
Serialise one SSE frame (data is single-line compact JSON).
Parse a Last-Event-ID header; anything but a positive integer means "from start".
Prefer: wait=N (RFC 7240) capped at :data:MAX_PREFER_WAIT_SECONDS; else None.
Validate an Idempotency-Key header value (raises ValueError).
Turn a request-validation location (("body", "kind")) into an RFC 6901 pointer
(/body/kind) for Issue.pointer in 422 SCHEMA_INVALID problems.
A time-sortable id <prefix>_<26 Crockford base32 chars> (ULID layout, lowercase).
The first 10 characters encode the millisecond timestamp, so ids sort by creation time (cursor pagination relies on it); the last 16 are 80 random bits.
Tenancy and authorization¶
See Tenancy and authentication.
Tenants, projects, roles, principals and authorization (plan §2, §8.1, §9).
- A tenant is the billing, isolation and quota boundary, with a :class:
TrustTierthat decides where its runs may be placed (:func:placement_allows) and a data-residency region (:func:region_satisfies). - A project is a workspace inside a tenant and the RBAC unit. Roles are ordered
viewer < editor < runner < admin: editors change datasets and specs, runners may also spend compute (runs, scores), admins manage members, endpoints and plugins. - :data:
PERMISSIONSis the whole permission matrix; :func:canis the pure check every component uses. :class:AuthContextcombines it with token scopes, tenant status and the operator flag (:meth:AuthContext.allows). - :func:
resolve_project_scopeturns theX-Mapi-Projectheader, path parameters and a token's binding into the(tenant, project)a request acts in.
Pure Pydantic and stdlib: the server, the web app (through OpenAPI) and tests share it.
ResidencyRegion = Annotated[
str,
Field(
pattern="^(?:[a-z]{2,6}|[a-z]+-[a-z]+[0-9]{1,2})$",
examples=["eu", "us", "europe-west1"],
description="A multi-region (eu, us, asia) or one cloud region (europe-west1)",
),
]
TIER_PLACEMENT: Final[
dict[TrustTier, PlacementRequirement]
] = {
TrustTier.INTERNAL: PlacementRequirement(
Isolation.PROCESS, False, False
),
TrustTier.SELF_HOSTED_TRUSTED: PlacementRequirement(
Isolation.PROCESS, False, False
),
TrustTier.VERIFIED: PlacementRequirement(
Isolation.GVISOR, True, True
),
TrustTier.PUBLIC: PlacementRequirement(
Isolation.GVISOR, True, True
),
}
Placement is enforced in dispatcher code, never by tenant configuration. Shared,
multi-tenant tiers (verified, public) need isolation ≥ gVisor with egress and DNS
enforced, exactly :meth:ExecutorCaps.allows_public_tenants.
PERMISSIONS: Final[
dict[tuple[ResourceKind, Action], Rule]
] = {
(_RK.TENANT, _AC.READ): Rule(_V, _T),
(_RK.TENANT, _AC.UPDATE): Rule(_A, _T),
(_RK.TENANT, _AC.CREATE): _OP,
(_RK.TENANT, _AC.DELETE): _OP,
(_RK.TENANT, _AC.MANAGE): _OP,
(_RK.TENANT_MEMBER, _AC.READ): Rule(_V, _T),
(_RK.TENANT_MEMBER, _AC.CREATE): Rule(_A, _T),
(_RK.TENANT_MEMBER, _AC.UPDATE): Rule(_A, _T),
(_RK.TENANT_MEMBER, _AC.DELETE): Rule(_A, _T),
(_RK.PROJECT, _AC.READ): Rule(_V, _P),
(_RK.PROJECT, _AC.CREATE): Rule(_A, _T),
(_RK.PROJECT, _AC.UPDATE): Rule(_A, _P),
(_RK.PROJECT, _AC.DELETE): Rule(_A, _T),
(_RK.MEMBER, _AC.READ): Rule(_V, _P),
(_RK.MEMBER, _AC.CREATE): Rule(_A, _P),
(_RK.MEMBER, _AC.UPDATE): Rule(_A, _P),
(_RK.MEMBER, _AC.DELETE): Rule(_A, _P),
(_RK.TOKEN, _AC.READ): Rule(_V, _P),
(_RK.TOKEN, _AC.CREATE): Rule(_V, _P),
(_RK.TOKEN, _AC.DELETE): Rule(_V, _P),
(_RK.TOKEN, _AC.MANAGE): Rule(_A, _P),
(_RK.CATALOG, _AC.READ): Rule(_V, _P),
(_RK.DATASET, _AC.READ): Rule(_V, _P),
(_RK.DATASET, _AC.CREATE): Rule(_E, _P),
(_RK.DATASET, _AC.UPDATE): Rule(_E, _P),
(_RK.DATASET, _AC.DELETE): Rule(_A, _P),
(_RK.SPEC, _AC.READ): Rule(_V, _P),
(_RK.SPEC, _AC.CREATE): Rule(_E, _P),
(_RK.SPEC, _AC.UPDATE): Rule(_E, _P),
(_RK.SPEC, _AC.DELETE): Rule(_A, _P),
(_RK.RUN, _AC.READ): Rule(_V, _P),
(_RK.RUN, _AC.CREATE): Rule(_R, _P),
(_RK.RUN, _AC.CANCEL): Rule(_R, _P),
(_RK.RUN, _AC.UPDATE): Rule(_R, _P),
(_RK.MODEL, _AC.READ): Rule(_V, _P),
(_RK.MODEL, _AC.RUN): Rule(_R, _P),
(_RK.MODEL, _AC.DELETE): Rule(_A, _P),
(_RK.ENDPOINT, _AC.READ): Rule(_V, _P),
(_RK.ENDPOINT, _AC.RUN): Rule(_R, _P),
(_RK.ENDPOINT, _AC.CREATE): Rule(_A, _P),
(_RK.ENDPOINT, _AC.UPDATE): Rule(_A, _P),
(_RK.ENDPOINT, _AC.PROMOTE): Rule(_A, _P),
(_RK.ENDPOINT, _AC.DELETE): Rule(_A, _P),
(_RK.ENDPOINT, _AC.MANAGE): Rule(_A, _P),
(_RK.PLOT, _AC.READ): Rule(_V, _P),
(_RK.PLOT, _AC.CREATE): Rule(_E, _P),
(_RK.PLOT, _AC.RUN): Rule(_R, _P),
(_RK.PLOT, _AC.DELETE): Rule(_E, _P),
(_RK.EMBED, _AC.READ): Rule(_V, _P),
(_RK.EMBED, _AC.CREATE): Rule(_E, _P),
(_RK.EMBED, _AC.DELETE): Rule(_E, _P),
(_RK.WEBHOOK, _AC.READ): Rule(_A, _P),
(_RK.WEBHOOK, _AC.CREATE): Rule(_A, _P),
(_RK.WEBHOOK, _AC.UPDATE): Rule(_A, _P),
(_RK.WEBHOOK, _AC.DELETE): Rule(_A, _P),
(_RK.PLUGIN, _AC.READ): Rule(_V, _P),
(_RK.PLUGIN, _AC.CREATE): Rule(_A, _P),
(_RK.PLUGIN, _AC.DELETE): Rule(_A, _P),
(_RK.USAGE, _AC.READ): Rule(_V, _P),
(_RK.BILLING, _AC.READ): Rule(_A, _T),
(_RK.AUDIT, _AC.READ): Rule(_A, _T),
(_RK.QUOTA, _AC.READ): _OP,
(_RK.QUOTA, _AC.UPDATE): _OP,
(_RK.QUOTA, _AC.DELETE): _OP,
(_RK.KILL_SWITCH, _AC.READ): _OP,
(_RK.KILL_SWITCH, _AC.UPDATE): _OP,
(_RK.EXECUTOR, _AC.READ): _OP,
(_RK.EXECUTOR, _AC.UPDATE): _OP,
(_RK.EVENT, _AC.CREATE): Rule(None, Level.INTERNAL),
(_RK.TENANT_DELETION, _AC.READ): Rule(_A, _T),
(_RK.TENANT_DELETION, _AC.CREATE): Rule(_A, _T),
(_RK.TENANT_DELETION, _AC.CANCEL): Rule(_A, _T),
(_RK.TENANT_DELETION, _AC.MANAGE): _OP,
(_RK.TENANT_EXPORT, _AC.READ): Rule(_A, _T),
(_RK.TENANT_EXPORT, _AC.CREATE): Rule(_A, _T),
(_RK.WORKFLOW, _AC.READ): Rule(_V, _P),
(_RK.WORKFLOW, _AC.CREATE): Rule(_E, _P),
(_RK.WORKFLOW, _AC.UPDATE): Rule(_E, _P),
(_RK.WORKFLOW, _AC.DELETE): Rule(_A, _P),
(_RK.WORKFLOW, _AC.RUN): Rule(_R, _P),
(_RK.WORKFLOW, _AC.CANCEL): Rule(_R, _P),
(_RK.POLICY, _AC.READ): Rule(_V, _P),
(_RK.POLICY, _AC.CREATE): Rule(_A, _P),
(_RK.POLICY, _AC.UPDATE): Rule(_A, _P),
(_RK.POLICY, _AC.DELETE): Rule(_A, _P),
(_RK.TENANT_POLICY, _AC.READ): Rule(_V, _T),
(_RK.TENANT_POLICY, _AC.CREATE): Rule(_A, _T),
(_RK.TENANT_POLICY, _AC.UPDATE): Rule(_A, _T),
(_RK.TENANT_POLICY, _AC.DELETE): Rule(_A, _T),
(_RK.MODEL, _AC.VALIDATE): Rule(_R, _P),
(_RK.MODEL, _AC.APPROVE): Rule(
_E, _P, Capability.REVIEWER
),
(_RK.MODEL, _AC.UPDATE): Rule(_A, _P),
}
The complete matrix. A pair that is not listed is denied to everyone.
OPERATOR_PERMISSIONS: Final[
frozenset[tuple[ResourceKind, Action]]
] = frozenset(
{
k
for k, rule in PERMISSIONS.items()
if rule.level is Level.PLATFORM
}
| {
(r, Action.READ)
for r in (
ResourceKind.TENANT,
ResourceKind.TENANT_MEMBER,
ResourceKind.PROJECT,
ResourceKind.MEMBER,
ResourceKind.TOKEN,
ResourceKind.RUN,
ResourceKind.USAGE,
ResourceKind.BILLING,
ResourceKind.AUDIT,
ResourceKind.CATALOG,
ResourceKind.TENANT_DELETION,
)
}
)
What platform operators may do. Operators read metadata (tenants, members, token listings, runs, usage, audit) but never tenant content (datasets, specs, models, plots).
Principal = Annotated[
UserPrincipal | TokenPrincipal | ServicePrincipal,
Field(discriminator="kind"),
]
Selects the project of a request: <tenant_id>/<project_id> (or a bare project id
when the tenant is unambiguous). See :func:resolve_project_scope.
Bases: ResponseModel
The billing, isolation and quota boundary.
Bases: ResponseModel
A workspace inside a tenant (M1: only default).
Bases: StrEnum
How much the platform trusts a tenant's code and data (plan §2, §7, §9).
Bases: StrEnum
suspended tenants are read-only; deleting tenants are inaccessible.
Creating, changing or running anything (including spending compute).
PlacementRequirement(
min_isolation: Isolation,
egress_enforced: bool,
dns_enforced: bool,
)
What an executor must provide before it may run a tier's work (plan §8.4).
Bases: StrEnum
Project roles, ordered viewer < editor < runner < admin (each includes the last).
Bases: StrEnum
Which grant a permission is checked against.
The effective role in the project (max of tenant-wide and project membership).
Rule(
min_role: ProjectRole | None,
level: Level,
capability: Capability | None = None,
)
M3.5: also needs this capability (project admins hold every capability).
Bases: ResponseModel
A role grant. project_id=None grants the role on the tenant and every project.
Bases: StrictModel
A person authenticated by OIDC. (issuer, subject) is the stable identity; the
platform id usr_… is assigned at first login.
Bases: StrictModel
A request authenticated with an API token (acts as owner_id, narrowed by scopes).
Bases: StrictModel
An internal platform component (bypasses RLS through its own database role).
Bases: StrEnum
Google Pub/Sub delivering to /internal/events (OIDC-authenticated).
Bases: StrictModel
Who is calling, in which tenant/project, with which grants (one per request).
role is the effective project role and tenant_role the tenant-wide role, both
resolved by the server from memberships (for a token: its owner's memberships now).
scopes is None for browser sessions (no narrowing) and the token's scopes else.
M3.5: capabilities held in the project (from the memberships).
allows(
action: Action | str,
resource: ResourceKind | str,
*,
resource_name: str | None = None,
) -> bool
The single authorization decision (pure; no I/O).
- internal services may do exactly the
internalrules; - operators may do :data:
OPERATOR_PERMISSIONS(regardless of tenant status); - otherwise the tenant must allow it (suspended: reads and
:data:
SUSPENDED_ALLOWEDonly; deleting: nothing), the role at the rule's level must satisfy :func:can, and a token's scopes must cover it (:func:mapi_spec.tokens.scope_allows).
require(
action: Action | str,
resource: ResourceKind | str,
*,
resource_name: str | None = None,
) -> None
Raise FORBIDDEN unless :meth:allows.
A parsed X-Mapi-Project value; tenant_id is None for a bare project id.
Can data with residency be processed in region?
A single-region residency needs exactly that region; a multi-region residency
(:data:MULTI_REGIONS) any region with a matching prefix. local (an executor with
no cloud region, e.g. local-process) never satisfies a residency other than itself.
placement_allows(
tier: TrustTier,
caps: ExecutorCaps,
*,
residency: str | None = None,
region: str | None = None,
) -> bool
May an executor with caps run work of a tier tenant (optionally in region,
for data that must stay in residency)?
required_rule(
action: Action | str, resource: ResourceKind | str
) -> Rule | None
The rule for (resource, action), or None if nobody may do it.
can(
role: ProjectRole | str | None,
action: Action | str,
resource: ResourceKind | str,
capabilities: Collection[Capability | str] = (),
) -> bool
Does role (with capabilities) permit action on resource?
role must be the role at the rule's :class:Level (see :func:effective_role):
the tenant-wide role for tenant-level rules, the effective project role otherwise.
Platform and internal rules are never granted by a role. A rule with a
:class:Capability also needs it, which project admins always hold.
effective_role(
memberships: Iterable[Membership],
principal_id: str,
tenant_id: str,
project_id: str | None = None,
) -> ProjectRole | None
The highest role principal_id holds: tenant-wide grants always count, project
grants only when project_id is given and matches.
parse_project_selector(value: str) -> ProjectSelector
Parse "<tenant>/<project>" or "<project>" (raises SCHEMA_INVALID).
resolve_project_scope(
*,
header: str | None = None,
path_tenant: str | None = None,
path_project: str | None = None,
token_binding: tuple[str, str] | None = None,
member_tenants: Collection[str] = (),
default: tuple[str, str] | None = None,
) -> ProjectSelector
Decide which (tenant, project) a project-scoped request acts in.
Precedence: path parameters, then the X-Mapi-Project header, then the token's
binding, then default (the single-tenant deployment profile only).
- path and header that disagree →
SCHEMA_INVALID; - a token used outside its binding →
FORBIDDEN(tokens are bound to one project); - a bare project id is qualified with the path tenant, the token's tenant, or the only
tenant the principal belongs to; otherwise →
SCHEMA_INVALID; - nothing selects a project and there is no default →
SCHEMA_INVALID.
Membership is checked afterwards; a project the caller cannot see is NOT_FOUND
(not FORBIDDEN), so ids cannot be probed.
API tokens¶
API tokens: wire format, scopes, at-rest hashing and the token DTOs (plan §8.2, §9).
Token format (all lowercase Crockford base32, 0-9a-z without i l o u)::
mapi_<prefix>_<secret><checksum>
8 chars 32 chars 7 chars
prefix(40 random bits) is public: it is shown in listings and is the lookup key;secret(160 random bits) is shown exactly once, in :class:TokenCreated;checksumis the CRC-32 ofmapi_<prefix>_<secret>(ASCII) in 7 base32 digits, so clients and secret scanners can reject typos and random strings without a database call.
At rest the server stores only :func:hash_token of the whole token (HMAC-SHA256 under a
server-side pepper) and compares with :func:verify_token_hash in constant time. No API
response ever contains the hash or, after creation, the secret.
A token acts as its owner, narrowed by its scopes: every request needs both the owner's
project role (:func:mapi_spec.tenancy.can) and a scope that covers the action
(:func:scope_allows). Tokens are bound to one project of one tenant.
TokenScope = Annotated[
str,
Field(
max_length=140,
examples=[
"models:read",
"runs:create",
"score:geo-mmm-prod",
],
),
AfterValidator(check_scope),
]
models:read | runs:create | score:{endpoint} | plugins:publish |
datasets:write | endpoints:promote | webhooks:manage | workflows:write |
workflows:run | models:validate | policies:manage | runs:write | admin.
The parts of a well-formed token. str(parsed) is the full token again.
Bases: StrEnum
Scope families. score is always written with a target: score:<endpoint alias>
or score:<model id> (score:*: every endpoint and model of the project).
M3: promote, roll back and delete endpoints (CI pipelines); never overrides a gate.
M3: manage the project's webhooks and replay deliveries.
M3.5: create workflows and push revisions (workflow-as-code from CI).
M3.5: start and cancel workflow runs, manage schedules and triggers.
M3.5: submit model versions for governance validation; never approves.
M3.5: manage the project's governance policies (policy-as-code).
v0.11: rename runs, edit notes and tags, log metrics and params to existing runs.
Bases: RequestModel
POST /v1/tokens: mint a token for the caller in the selected project.
The token can never do more than its owner's role allows at the time of use. Limits are
optional: None means the tenant's defaults apply (they always apply in addition).
Bases: ResponseModel
A token as listed by the API. It never carries the secret or its hash.
Bases: ResponseModel
201 body of POST /v1/tokens: the only time secret is ever returned.
Bases: StrEnum
Bases: StrEnum
Window of a per-token vCPU-second budget (UTC calendar day or month).
generate_token(prefix: str | None = None) -> ParsedToken
A new random token (160-bit secret). prefix is for tests and migrations only.
parse_token(value: str) -> ParsedToken
Split and checksum-verify a token. Raises UNAUTHENTICATED on any defect.
The error never echoes the presented value (it may be a real secret with a typo).
The 7-character checksum of mapi_<prefix>_<secret> (CRC-32, base32).
True if text contains a checksum-valid token (for secret scanning and log scrubbing).
mapi_<prefix>_…: the only form of a token that may be displayed or logged.
Extract the credential from an Authorization: Bearer <token> header value.
hash_token(token: str | ParsedToken, pepper: bytes) -> str
At-rest digest of a whole token: hmac-sha256:<64 hex> under the server pepper.
verify_token_hash(
token: str | ParsedToken, stored: str, pepper: bytes
) -> bool
Constant-time comparison of a presented token with its stored :func:hash_token.
parse_scope(value: str) -> tuple[ScopeName, str | None]
"score:geo-mmm-prod" → (SCORE, "geo-mmm-prod"); other scopes have no target.
score:* covers every endpoint of the token's project.
scope_allows(
scopes: Iterable[str],
action: str,
resource: str,
*,
resource_name: str | None = None,
) -> bool
Does any of scopes cover action on resource?
action/resource are :class:mapi_spec.tenancy.Action /
:class:mapi_spec.tenancy.ResourceKind values (plain strings here so this module does
not depend on :mod:mapi_spec.tenancy). resource_name is the endpoint alias (or the
model id) for score:<target>. Scopes only narrow: the owner's role is checked
separately.
===================== =======================================================
scope covers
===================== =======================================================
admin everything the owner's role allows
models:read read on catalog, dataset, spec, run, model, endpoint,
plot, usage, project
runs:create models:read + create/cancel run, run
model, endpoint, plot
score:<target> read + run on that endpoint and run (score)
on the model with that id (*: every endpoint and model)
datasets:write read/create/update dataset
plugins:publish read/create/delete plugin
endpoints:promote read/create/update/promote/delete
endpoint, read model
webhooks:manage read/create/update/delete webhook
workflows:write read/create/update workflow, read catalog
workflows:run read/run/cancel/update workflow (schedules,
triggers), read/create/cancel run, run
model, read dataset
models:validate read/validate model, read policy, read/
create run (the governance checks run)
policies:manage read/create/update/delete policy
runs:write models:read + update run (name, note, tags, metric
and param writes)
===================== =======================================================
No scope covers :data:TOKEN_NEVER: minting tokens (a child token would escape its
parent's expiry, budget, rate limit and revocation), overriding a promotion gate and
approving a model validation (human decisions that are audited) need a signed-in
session.
Sessions and Pub/Sub push¶
Authentication protocols shared by the server and the web app (plan §8.1, §8.7, §11).
Browser sessions (BFF). The SPA never sees an OAuth token. GET /auth/login starts an
OpenID Connect Authorization Code flow with PKCE (S256) and a nonce; the transaction is
kept server-side, referenced by a short-lived __Host- cookie. GET /auth/callback
checks state, exchanges the code, verifies the ID token (issuer, audience, nonce, expiry)
and sets the session cookie: HttpOnly; Secure; SameSite=Strict; Path=/ with no
Domain. Because the cookie is SameSite=Strict and every state-changing endpoint takes
JSON (a CORS-preflighted content type), cross-site requests never carry it.
Programs send Authorization: Bearer mapi_… (see :mod:mapi_spec.tokens).
Pub/Sub push to POST /internal/events carries a Google-signed OIDC ID token; after
verifying the signature, the server checks its claims with :func:check_push_claims.
Bases: StrictModel
Server-side state of one login between /auth/login and /auth/callback.
Bases: ResponseModel
GET /auth/session: is the browser signed in, as whom, until when.
Bases: ResponseModel
The message of a Pub/Sub push request (both field spellings are accepted).
The decoded payload (raises ValueError for bad base64 or an oversized body).
Bases: ResponseModel
Body of a Pub/Sub push to POST /internal/events.
Unknown fields are ignored (Google adds fields over time). Authentication is the OIDC
token in Authorization, never anything in the body.
Bases: StrictModel
What the ID token on a Pub/Sub push must say (checked after signature verification).
A fresh RFC 7636 code_verifier (43-128 unreserved characters).
The S256 code_challenge of verifier: BASE64URL(SHA256(verifier)).
A same-origin path to send the browser to after login (blocks open redirects).
Only absolute paths are accepted: no scheme, no host, no // or /\ prefix,
no control characters. Anything else yields default.
build_authorize_url(
authorization_endpoint: str,
*,
client_id: str,
redirect_uri: str,
tx: LoginTransaction,
scopes: Sequence[str] = OIDC_SCOPES,
extra: Mapping[str, str] | None = None,
) -> str
The IdP authorization URL for tx (Authorization Code + PKCE S256 + nonce).
check_push_claims(
claims: Mapping[str, Any],
expect: PushAuthExpectation,
*,
now: datetime,
) -> list[str]
Problems with a push token's claims (empty list: accept). Signature, key rotation and JWKS fetching are the verifier library's job; this checks what the token asserts.
Quotas and scheduling¶
See Quotas and cost.
Quotas, rate limits, priority classes and fair-share leasing (plan §8.3, §8.8, §9).
Three levels of :class:QuotaPolicy apply to every request and run, most specific first:
token, tenant, global. Each sets optional :class:QuotaLimits (None =
unlimited at that level). :func:evaluate_quota is the single pure decision function: the
first limit a demand would exceed yields a :class:QuotaDecision with problem code
RATE_LIMITED (request rate; retryable) or QUOTA_EXCEEDED (everything else).
Quotas are expressed in vCPU-seconds because abuse risk is compute, not run count.
:class:PriorityClass weights drive :func:plan_leases, the dispatcher's weighted
fair-share choice of which queued runs to lease next, so one tenant's 237-task SBC cannot
starve everyone else's interactive scoring.
DEFAULT_TIER_LIMITS: Final[dict[TrustTier, QuotaLimits]] = {
TrustTier.PUBLIC: QuotaLimits(
vcpu_seconds_per_day=3600,
vcpu_seconds_per_month=36000,
concurrent_runs=2,
max_fanout_tasks=16,
storage_bytes=2 * GIB,
egress_bytes_per_month=5 * GIB,
requests_per_minute=60,
max_resource_class=ResourceClass.S,
),
TrustTier.VERIFIED: QuotaLimits(
vcpu_seconds_per_day=86400,
vcpu_seconds_per_month=1000000,
concurrent_runs=20,
max_fanout_tasks=256,
storage_bytes=100 * GIB,
egress_bytes_per_month=100 * GIB,
requests_per_minute=600,
max_resource_class=ResourceClass.L,
),
TrustTier.INTERNAL: QuotaLimits(
concurrent_runs=100,
max_fanout_tasks=1000,
requests_per_minute=6000,
),
TrustTier.SELF_HOSTED_TRUSTED: QuotaLimits(),
}
Tenant limits when no tenant policy overrides them (free tier: class S only, plan Q6).
DEFAULT_GLOBAL_LIMITS: Final = QuotaLimits(
concurrent_runs=700
)
70% of Cloud Run's 1,000 running executions per project and region (plan §8.8).
PRIORITY_WEIGHTS: Final[dict[PriorityClass, float]] = {
PriorityClass.INTERACTIVE: 4.0,
PriorityClass.BATCH: 1.0,
}
Fair-share weights: at equal load an interactive run is leased before a batch run, and a tenant's running batch work counts four times as heavily against its next interactive run.
Bases: StrEnum
Policy level, most specific first.
Bases: StrictModel
Limits at one level. None = no limit at this level (other levels still apply).
tightened(other: QuotaLimits) -> QuotaLimits
Field-wise the more restrictive of self and other.
Bases: ResponseModel
The limits in force for one subject at one level.
Bases: RequestModel
PUT /v1/admin/quotas: create or replace the policy for (scope, subject_id).
Bases: StrictModel
Counters for one subject in the current windows (UTC day, UTC month, minute).
Bases: StrictModel
What a request or submission is about to consume (estimates for compute).
Bases: ResponseModel
The outcome of :func:evaluate_quota. allowed=False carries the first limit hit.
Bases: StrEnum
Every limit a :class:QuotaDecision can name.
Bases: ResponseModel
One limit of one level as shown in /usage (remaining is None if unlimited).
Bases: StrEnum
Scheduling class of a run. Interactive work (scores, renders) outranks batch fits.
LeaseCandidate(
run_id: str,
tenant_id: str,
priority: PriorityClass,
queued_at: datetime,
tasks: int = 1,
)
A queued run the dispatcher could lease (tasks > 1 for taskCount fan-out).
evaluate_quota(
levels: Iterable[tuple[QuotaPolicy, QuotaUsage]],
demand: QuotaDemand,
*,
now: datetime,
) -> QuotaDecision
Check demand against every level; return the first violation or an allowance.
Order: request rate at every level first (RATE_LIMITED), then per level from token to
global (QUOTA_EXCEEDED): resource class, fan-out width, concurrent runs, vCPU-seconds
per day and month, storage, egress. A cumulative limit fails when
used + requested > limit and only if the demand asks for some of it, so a subject
already over its storage quota can still read and delete. retry_after_seconds is set
for windowed limits (minute, UTC day, UTC month).
quota_status(
policy: QuotaPolicy, usage: QuotaUsage, *, now: datetime
) -> list[QuotaLimitStatus]
The cumulative limits of one level with their current use (for /usage).
Start of the next UTC window after now.
policy_for_token(record: TokenRecord) -> QuotaPolicy
The token-level policy implied by a token's own rate limit and vCPU budget.
default_priority(kind: RunKind | str) -> PriorityClass
Interactive for short, user-facing kinds (score, render, derive, validate, describe, translate_priors); batch for fits, ingest, evaluate, optimise and fan-out parents.
plan_leases(
candidates: Sequence[LeaseCandidate],
*,
capacity: int,
running_tasks: Mapping[str, int] | None = None,
concurrency_left: Mapping[str, int | None]
| None = None,
tenant_weights: Mapping[str, float] | None = None,
weights: Mapping[
PriorityClass, float
] = PRIORITY_WEIGHTS,
) -> list[LeaseCandidate]
Choose which queued runs to lease now (weighted fair share; pure and deterministic).
Repeatedly picks the candidate with the smallest virtual finish
(load[tenant] + tasks) / (tenant_weight × priority_weight), ties broken by
queued_at then run_id. load starts at the tenant's running_tasks and grows
with every pick. A candidate is skipped when its tasks exceed the remaining global
capacity or its tenant has no concurrency_left (runs). Returns the picks in order.
backoff_seconds(
attempt: int,
*,
base: float = 1.0,
cap: float = 64.0,
retry_after: float | None = None,
) -> float
Delay before retry attempt (1-based) after a 429: min(cap, base·2^(attempt-1)),
never shorter than a server-sent Retry-After. Callers add jitter.
Cost¶
Price tables, cost estimates and the per-run cost ledger (plan §8.2, §13 cost floor, §14 M2).
Money is integer micros of a currency (1 USD = 1,000,000 micros); unit prices are
integer nanos per vCPU-second and per GiB-second, so every amount is exact and
reproducible from a ledger row (round half to even at the micro). Every amount names the
:class:PriceTable version it was priced with; changing prices never rewrites history.
Usage of an attempt is vcpu × billed_seconds × tasks vCPU-seconds and
ram_gib × billed_seconds × tasks GiB-seconds, from :data:RESOURCE_CLASSES;
billed_seconds is wall time rounded up to the table's billing increment. A class with
accelerators (GPU) also meters accelerators × billed_seconds × tasks accelerator-seconds.
:data:DEFAULT_PRICE_TABLE mirrors Cloud Run's published list prices (CPU 0.000018 USD per
vCPU-second, memory 0.000002 USD per GiB-second); deployments configure their own. An E33
fit on class M for five minutes costs about 0.03 USD.
DEFAULT_PRICE_TABLE: Final = PriceTable(
version="cloudrun-list-2026-10",
currency="USD",
classes={
ResourceClass.S: _LIST,
ResourceClass.M: _LIST,
ResourceClass.L: _LIST,
ResourceClass.GPU: _GPU,
},
billing_increment_ms=100,
source="Cloud Run list prices (CPU per vCPU-second, memory per GiB-second); a default for estimates, configure the deployment's own table",
)
Cloud Run-like list-price defaults. XL (GKE, M6) has no default price; GPU adds
the L4 accelerator price.
Bases: StrictModel
A versioned, immutable price list. Classes without a row cannot be priced.
Wall time rounded up to the billing increment (and the minimum).
Bases: StrictModel
Unit prices for one resource class, in nanos of the table's currency.
Bases: StrictModel
Bases: ResponseModel
A pre-run estimate (shown before submit; checked against vCPU quotas).
Bases: StrEnum
actual: metered from an attempt; adjustment: an operator correction (may be
negative). Estimates are never written to the ledger.
Bases: ResponseModel
One append-only ledger row: what one attempt of one run cost.
Bases: ResponseModel
Sums over ledger entries (one currency).
Bases: ResponseModel
Cost of a run (all attempts). estimated is true until an actual ledger entry
exists; usd is kept for v0.2 clients and is set only when the currency is USD.
from_ledger(
entries: Iterable[CostLedgerEntry],
*,
estimate: CostEstimate | None = None,
) -> RunCost
Fold a run's ledger entries (and its pre-run estimate) into the Run DTO field.
class_usage(
resource_class: ResourceClass,
wall_seconds: float,
*,
task_count: int = 1,
table: PriceTable = DEFAULT_PRICE_TABLE,
) -> ResourceUsage
Billable usage of task_count tasks of resource_class for wall_seconds each.
price_usage(
usage: ResourceUsage,
resource_class: ResourceClass,
table: PriceTable = DEFAULT_PRICE_TABLE,
) -> int
Amount in micros (round half to even).
estimate_cost(
resource_class: ResourceClass,
wall_seconds: float,
*,
task_count: int = 1,
table: PriceTable = DEFAULT_PRICE_TABLE,
) -> CostEstimate
Price task_count tasks of resource_class running wall_seconds each.
ledger_entry(
*,
tenant_id: str,
project_id: str,
run_id: str,
attempt: int,
resource_class: ResourceClass,
wall_seconds: float,
recorded_at: datetime,
task_count: int = 1,
executor: str | None = None,
token_id: str | None = None,
table: PriceTable = DEFAULT_PRICE_TABLE,
entry_id: str | None = None,
) -> CostLedgerEntry
Meter and price one finished attempt (actual entry).
summarize_ledger(
entries: Iterable[CostLedgerEntry],
group_by: GroupBy | None = None,
) -> dict[str, CostTotals]
Totals overall (key "total") or per group (day/month are UTC dates).
31200, "USD" → "0.031200 USD" (exact; no float rounding).
Audit¶
Audit events: who did what to which resource, and whether it worked (plan §9, §11 Admin).
Audit rows are append-only. They identify actors by platform id, never by e-mail, and never
store raw client IPs or user agents: :func:client_fingerprint keys an HMAC with a
deployment secret, so rows can be correlated ("same address as before") without being
reversible. Secrets (token values, signed URLs) never appear in details.
Bases: StrEnum
Audited actions (<resource>.<verb>). New actions are additive.
Conformance or security-suite results recorded for an executor (shared-tier gate).
A pushed run event failed HMAC or identity checks (a security signal).
A self-service sign-up was accepted (a verification e-mail was sent).
A sign-up was refused (disposable domain, terms not accepted, rate limit).
A verified sign-up tripped a Sybil heuristic and waits for review (it still works).
An operator approved or rejected a sign-up (a rejection suspends its tenant).
An operator recorded (or revoked) identity verification of a tenant.
An operator restarted a failed tenant purge.
Every store of a deleted tenant was purged (kept without content, by tenant id).
A promotion went ahead over a warn gate: actor, reason and failing checks.
The platform disabled a webhook (too many failures, or 410 Gone).
A webhook URL resolved to a blocked address (an SSRF attempt or a misconfiguration).
A new workflow revision was saved (with its validation result).
A workflow run started (manually, on a schedule, by a trigger or a validation).
A new policy version (the previous version is kept).
Every change of a model version's validation status (from, to, event, actor).
A promotion was refused because the policy requires a validated model.
PATCH /v1/runs/{id}: details {"fields": [...], "tags_set": n, "tags_deleted": n}
(never the note text).
Metrics, params or media written through the API: details {"metrics": n} /
{"params": n} / {"media": 1, "kind": "image"}.
Rename, description, archive, or a bulk run move (details {"added": n, "removed": n}).
Bases: StrEnum
Authentication or authorization refused it (including quota and kill switches).
Bases: ResponseModel
One audit row. tenant_id is None for platform-level actions.
Bases: RequestModel
Filters of GET /v1/admin/audit and GET /v1/tenants/{tenant_id}/audit
(sent as query parameters; tenant audits force tenant_id).
Keyed, truncated digest of a client IP or user agent (None stays None).
make_audit_event(
*,
action: AuditAction,
resource_kind: ResourceKind,
outcome: AuditOutcome,
ts: datetime,
fingerprint_key: bytes,
tenant_id: str | None = None,
project_id: str | None = None,
actor_kind: PrincipalKind | None = None,
actor_id: str | None = None,
token_id: str | None = None,
operator: bool = False,
resource_id: str | None = None,
code: ErrorCode | None = None,
client_ip: str | None = None,
user_agent: str | None = None,
request_id: str | None = None,
details: Mapping[str, JsonValue] | None = None,
) -> AuditEvent
Build an :class:AuditEvent, hashing the client IP and user agent on the way in.
Operator controls¶
Operator controls: kill switches and executor health (plan §9 denial of wallet, §11 Admin).
:class:KillSwitches is a single platform-wide document. :func:kill_switch_decision is the
pure check the API (at submit) and the dispatcher (at lease) both apply: when
untrusted_execution_enabled is off, no run of uploaded (tenant) code is accepted or
leased; when the global spend in the current window reaches global_spend_cap_micros,
no new compute is accepted for anyone except internal tenants. Both denials use the
QUOTA_EXCEEDED problem code at the global scope.
Bases: ResponseModel
Platform-wide emergency controls (operators only; every change is audited).
Bases: RequestModel
PUT /v1/admin/kill-switches: fields left out keep their value; reason is
required so the audit log says why.
apply(
current: KillSwitches, *, now: datetime, by: str | None
) -> KillSwitches
The switches after this update.
Bases: ResponseModel
One executor as the admin console shows it.
Bases: StrEnum
Disabled by an operator (PUT /v1/admin/executors/{name}): nothing is leased.
kill_switch_decision(
switches: KillSwitches,
*,
untrusted_code: bool,
tier: TrustTier,
spent_micros: int,
now: datetime,
) -> QuotaDecision
May a new run start? untrusted_code: the run executes tenant-uploaded plugin code
(T2/T3). spent_micros: platform spend in the current cap window (same currency).
Executor conformance suite¶
The executor conformance suite, as data (plan §8.4, §13 "Executor conformance").
An executor (local-process, docker, docker-gvisor, cloudrun-jobs, ...) is
enabled only after it passes every applicable case of :data:EXECUTOR_CONFORMANCE. Each
:class:ConformanceCase names the behaviour, the fault the harness injects, what the
backend must support for the case to apply, and the :class:Expectation on the run's
observed trajectory. The server's pytest suite parametrizes over this tuple and checks each
observation with :func:check_expectation; the admin API reports results as a
:class:ConformanceReport.
EXECUTOR_CONFORMANCE: Final[tuple[ConformanceCase, ...]] = (
ConformanceCase(
id=_C.SUBMIT,
title="Submit runs one execution to success",
behaviour="A submitted attempt starts with identifiers in the environment only, reports status through the executor API, emits verified events and succeeds.",
steps=(
"submit a tiny fit with a fresh nonce",
"poll status until terminal; collect events",
"assert argv carries no run id, nonce or credential",
),
expect=_E(
final_state=_S.SUCCEEDED,
must_visit=(
_S.PROVISIONING,
_S.RUNNING,
_S.FINALIZING,
),
attempts=1,
executions_created=1,
),
plan_refs=("§8.1", "§8.3", "§8.4"),
),
ConformanceCase(
id=_C.ADOPT_AFTER_DISPATCHER_CRASH,
title="Orphan adoption after a dispatcher crash",
behaviour="If the dispatcher dies between the executor accepting an attempt and the handle being stored, the reconciler adopts the execution by nonce instead of starting a second one.",
steps=(
"write the intent row, submit, crash before storing the handle",
"restart the orchestrator; reconciler calls adopt(nonce)",
"the adopted execution finishes the run",
),
fault=Fault.DISPATCHER_CRASH_AFTER_SUBMIT,
expect=_E(
final_state=_S.SUCCEEDED,
attempts=1,
executions_created=1,
),
plan_refs=("§8.3",),
),
ConformanceCase(
id=_C.DUPLICATE_SUBMIT,
title="Duplicate submit is safe",
behaviour="Submitting the same (run, attempt, nonce) twice never yields two live executions; the second call returns or adopts the first, or its execution is cancelled before it does any work.",
steps=(
"submit twice with the same nonce",
"wait for the run to finish",
),
fault=Fault.DUPLICATE_SUBMIT,
expect=_E(
final_state=_S.SUCCEEDED,
attempts=1,
max_live_executions=1,
),
plan_refs=("§8.3",),
),
ConformanceCase(
id=_C.CANCEL_MID_RUN,
title="Cancel while running",
behaviour="A cancel during RUNNING moves the run to CANCELLING; the supervisor gets SIGTERM and flushes; the run is CANCELLED only after the executor confirms the execution is gone.",
steps=(
"submit a long fit; wait for RUNNING",
"cancel; poll executor status",
"assert CANCELLED is recorded after the executor reports a terminal state",
),
fault=Fault.CANCEL_WHILE_RUNNING,
expect=_E(
final_state=_S.CANCELLED,
must_visit=(_S.RUNNING, _S.CANCELLING),
must_not_visit=(_S.SUCCEEDED, _S.FAILED),
),
plan_refs=("§8.3", "§8.7"),
),
ConformanceCase(
id=_C.TIMEOUT,
title="Timeout is enforced and classified",
behaviour="A run that exceeds its timeout is stopped by the backend (hard task timeout as backstop) and ends FAILED with failure class timeout.",
steps=(
"submit a run that sleeps past a short timeout",
"wait for the run to end",
),
fault=Fault.EXCEED_TIMEOUT,
expect=_E(
final_state=_S.FAILED,
failure_class=FailureClass.TIMEOUT,
attempts=1,
),
plan_refs=("§8.3", "§8.7"),
),
ConformanceCase(
id=_C.STALLED_NOT_LOST,
title="Event silence raises STALLED, never a false LOST",
behaviour="When events stop but the executor still reports the execution running, the run becomes STALLED (a warning) and returns to RUNNING when events resume; it is never failed or retried because of silence alone.",
steps=(
"submit; drop events beyond K x interval while the execution runs",
"resume events; let the run finish",
),
fault=Fault.EVENT_SILENCE,
expect=_E(
final_state=_S.SUCCEEDED,
must_visit=(_S.STALLED,),
must_not_visit=(_S.FAILED,),
attempts=1,
executions_created=1,
),
plan_refs=("§8.3", "Key decisions: liveness"),
),
ConformanceCase(
id=_C.CONFIRMED_CANCEL_BEFORE_RETRY,
title="Confirmed cancel before an infrastructure retry",
behaviour="After an infrastructure failure the reconciler cancels the old execution and waits for a confirmed terminal status before re-queueing attempt + 1, so two attempts never run at once.",
steps=(
"submit; report an infra failure while the old execution is alive",
"observe cancel → confirmed terminal → QUEUED(attempt 2)",
"attempt 2 succeeds",
),
fault=Fault.INFRA_FAILURE_LIVE_EXECUTION,
expect=_E(
final_state=_S.SUCCEEDED,
must_visit=(_S.QUEUED,),
attempts=2,
max_live_executions=1,
executions_created=2,
),
plan_refs=("§8.3",),
),
ConformanceCase(
id=_C.CHECKPOINT_RESUME,
title="Checkpoint, PARTIAL and resume",
behaviour="A SIGTERM after checkpoints were written yields a PARTIAL bundle; resume starts a new attempt from the checkpoints and completes without redoing them.",
steps=(
"submit a segmented fit; wait for the first checkpoint",
"SIGTERM the execution; expect PARTIAL",
"resume; expect SUCCEEDED with the earlier segments reused",
),
fault=Fault.SIGTERM_WITH_CHECKPOINTS,
requires=frozenset(
{Requirement.CHECKPOINT_STORAGE}
),
expect=_E(
final_state=_S.SUCCEEDED,
must_visit=(_S.PARTIAL,),
attempts=2,
),
plan_refs=("§8.3", "§5 segmented sampling"),
budget_seconds=300,
),
ConformanceCase(
id=_C.OOM_CLASSIFICATION,
title="Out-of-memory is classified as resource exhaustion",
behaviour="A worker killed for memory (exit 137) ends FAILED with failure class resource_exhausted and an offer to retry one class up.",
steps=(
"submit a memory bomb at class S",
"wait for the run to end",
),
fault=Fault.MEMORY_BOMB,
requires=frozenset({Requirement.MEMORY_LIMIT}),
expect=_E(
final_state=_S.FAILED,
failure_class=FailureClass.RESOURCE_EXHAUSTED,
attempts=1,
offers_next_class=True,
),
plan_refs=("§8.5",),
),
ConformanceCase(
id=_C.FANOUT_TASK_COUNT,
title="Fan-out with taskCount",
behaviour="One execution with task_count N runs N tasks, each mapping its task index to a child spec and writing under tasks/{i}/; status aggregates all tasks and a cancel stops all of them.",
steps=(
"submit task_count=4",
"assert 4 distinct task indexes and per-task output prefixes",
"assert one backend execution",
),
requires=frozenset({Requirement.FANOUT}),
expect=_E(
final_state=_S.SUCCEEDED,
attempts=1,
executions_created=1,
),
plan_refs=("§8.3", "§8.6"),
),
ConformanceCase(
id=_C.GENERATION_VERIFICATION,
title="Output generations are verified at FINALIZING",
behaviour="An output overwritten after the runner's manifest (new generation or hash) or an unexpected extra object fails the run with failure class integrity; nothing is copied into the bundles bucket.",
steps=(
"submit; after the complete event, overwrite one output object",
"let the finalizer run",
),
fault=Fault.TAMPER_OUTPUT_GENERATION,
expect=_E(
final_state=_S.FAILED,
failure_class=FailureClass.INTEGRITY,
must_visit=(_S.FINALIZING,),
must_not_visit=(_S.SUCCEEDED,),
),
plan_refs=("§8.6", "§9 artifact swap"),
),
ConformanceCase(
id=_C.RATE_LIMIT_BACKOFF,
title="429 back-off without duplicates",
behaviour="When the backend's control API answers 429, the dispatcher backs off (exponential, honouring Retry-After), raises the 429 alarm counter and submits exactly once when the limit clears.",
steps=(
"make the control API return 429 to the first 3 submit calls",
"assert growing delays between calls",
"assert one execution and a SUCCEEDED run",
),
fault=Fault.BACKEND_429,
requires=frozenset(
{Requirement.REMOTE_CONTROL_API}
),
expect=_E(
final_state=_S.SUCCEEDED,
attempts=1,
executions_created=1,
),
plan_refs=("§8.3", "§8.8"),
),
)
Every case, in suite order.
Bases: StrictModel
Bases: StrEnum
Bases: StrEnum
What the harness does to the system under test.
Kill the dispatcher after the executor accepted the attempt, before the handle is
stored (the intent row (run, attempt, nonce) exists).
Call submit twice with the same (run, attempt, nonce).
Drop events for longer than the STALLED threshold while the execution keeps running.
Report an infrastructure failure while the old execution is still alive.
Overwrite an output object after the runner's manifest was signed.
The backend's control API answers HTTP 429 to the first submissions.
Bases: StrictModel
Constraints on the observed trajectory of the run under test.
Bases: StrictModel
An executor under test: its declared caps plus facts the caps do not express.
Bases: StrictModel
What the harness saw while one case ran.
Bases: ResponseModel
One run of the suite against one executor.
passed_for(target: ConformanceTarget) -> bool
True if this report is for the current suite and every applicable case passed.
Bases: ResponseModel
applicable_cases(
target: ConformanceTarget,
) -> tuple[ConformanceCase, ...]
The cases target must pass (cases whose requirements it supports).
check_expectation(
case: ConformanceCase, observed: ObservedRun
) -> list[str]
Every way observed violates case.expect (empty list: the case passed).
Guided mode v0.4¶
See Guided mode.
Guided mode: templates, the question flow, role detection and drafts (plan §11; M3).
Guided mode is organised around tasks. The user picks a :class:GuidedTemplate ("What do
you want to learn?"), answers plain-language :class:GuidedQuestion s about their columns
("Which column is revenue?") - pre-filled by :func:suggest_roles from column profiles -
and gets a draft: an :class:~mapi_spec.ingest.IngestMapping for the upload plus a
ModelSpec document (and its YAML, "View as YAML"), built by :func:build_draft.
Templates are data. Their spec_template is a ModelSpec document with
{{placeholder}} strings, filled from answers, template defaults and lookups
(:func:render_template); every placeholder is covered by a question, a default, a lookup
or a value the builder always provides (:data:BUILDER_VALUES), which a test checks.
Built-in templates (:data:GUIDED_TEMPLATES):
================= ========================== =============================================
id plugin for
================= ========================== =============================================
geo_mmm mmm.bayesian marketing mix by market (what-if, priors;
mmm-framework's BayesianMMM)
mediated_mmm mmm.nested national marketing mix whose channels act
through measured intermediate results
(visits, awareness)
multi_kpi_mmm mmm.multivariate national marketing mix of several KPI
columns with cross-effects between them
campaign_impact````causal.impact the effect of a campaign or a change on one
series (CausalImpact on a Bayesian
structural time series, controls optional)
geo_lift causal.synthetic_controlthe effect of a campaign in test markets
against a weighted mix of the other markets
(its result is a lift test for the MMM)
weekly_forecast````ts.hsgp_trend one weekly series, forecast ahead
store_demand tabular.regressor demand per store and week (quantiles)
customer_choice````choice.multinomial_logitwhich product customers choose on each
trip (a conditional logit with household
effects, in plugins/tabular)
================= ========================== =============================================
GUIDED_TEMPLATES: Final[dict[str, GuidedTemplate]] = {
t.id: t.model_copy(
update={"about": TEMPLATE_ABOUTS[t.id]}
)
for t in (
GEO_MMM,
MEDIATED_MMM,
MULTI_KPI_MMM,
CAMPAIGN_IMPACT,
GEO_LIFT,
WEEKLY_FORECAST,
STORE_DEMAND,
CUSTOMER_CHOICE,
)
}
The built-in guided templates, by id (GET /v1/guided/templates).
Bases: ResponseModel
A guided task: questions, the spec they fill and what to check afterwards.
ui_levels maps JSON pointers of the spec to x-ui-level hints (hidden fields
are kept but not shown; Expert edits round-trip). plugin_available is false while
the template's built-in plugin is not shipped (plugin_note says what is missing).
A family this version does not know (added by a newer server) is kept as text.
Bases: ResponseModel
One plain-language question. Column questions name what the column becomes:
dim: the column labels a dataset dim (dim_specdescribes it); the placeholder{{<id>}}is the dataset dim name;role(column): the column is a data variable with that role; the placeholder is the dataset variable name;role+wide_dim(columns): the columns become one variableas_varalongwide_dim(one label per column, e.g. spend per channel);rolewithoutwide_dim(columns): each column is its own variable.
A text answer fills its placeholder as typed (stripped); with as_list it is split
at the commas into a list of names (a required one must keep a name).
Bases: ResponseModel
A placeholder derived from an answer: map[answer] (e.g. frequency → season).
Bases: RequestModel
What the browser (DuckDB-WASM/papaparse preview) or an ingest profile knows about a
column. name and sample are user data: render them as text only.
Bases: RequestModel
POST /v1/guided/detect-roles: exactly one of dataset (a stored version:
its schema and profile are used) or columns (profiled in the browser).
Bases: ResponseModel
Bases: ResponseModel
Candidates for one question (best first) and the pre-filled answer.
Bases: ResponseModel
Bases: RequestModel
POST /v1/guided/drafts: answers to a template's questions.
Column answers are column names of the upload (a list for columns questions).
dataset names the dataset the upload will become (dataset://<project>/<name>);
columns (optional) lets the draft use detected types and frequencies.
Bases: ResponseModel
The draft a template and its answers produce: how to ingest the upload, the spec, the spec as YAML ("View as YAML") and anything worth a second look.
get_template(template_id: str) -> GuidedTemplate
A built-in template (NOT_FOUND otherwise).
Every {{name}} in a template (keys and values).
Fill {{name}} placeholders: a string that is only a placeholder becomes the
value itself (any JSON); placeholders inside longer strings are replaced by the value's
text. Raises KeyError for a placeholder without a value.
suggest_roles(
template: GuidedTemplate,
columns: Sequence[ColumnProfile],
) -> RoleSuggestions
Rank columns for every column question and pre-fill answers (pure heuristics).
Single-column questions take the best column at confidence ≥ 0.5 not used by an earlier
question; multi-column questions take every remaining column at ≥ 0.5. frequency
is answered from the chosen date column's detected frequency.
build_draft(
template: GuidedTemplate,
answers: GuidedAnswers,
*,
project: str,
) -> GuidedDraft
Turn answers into an ingest mapping and a ModelSpec (pure; validated).
Raises GUIDED_ANSWERS_INVALID for a missing required answer, a column used twice,
a choice or number out of range, or a resulting spec that does not parse.
A dataset name for a column (Revenue (EUR) → revenue_eur).
A label from a wide column name (TV_Spend_EUR → tv): the name's words minus
money and unit words and minus noise (the question's keywords, so price_coke
and promo_coke both become coke).
Ingest presets v0.4¶
Ingest presets and file contracts (plan §3.2, §11 Guided task 2, Q8; M3).
An upload becomes a labelled dataset in a sandboxed ingest run. Besides an explicit
column mapping (:class:mapi_spec.api.IngestMapping), M3 adds presets for the formats
marketing teams already have:
================= ===========================================================================
preset file contract
================= ===========================================================================
none the explicit dims/vars/wide column mapping
mff the MFF long table (:data:MFF_COLUMNS): one row per period, key columns
and variable, the value in VariableValue (:class:MffOptions)
meridian Google Meridian's CoordToColumns wide table (:class:MeridianColumns)
================= ===========================================================================
Wide tables (one column per channel) become one variable along a new dim through
:class:WideVarMapping - the shape Meridian and most spreadsheets use.
This module only describes files; every byte of an upload is parsed inside the sandbox.
MFF_COLUMNS: Final[tuple[str, ...]] = (
"Period",
"Geography",
"Product",
"Outlet",
"Campaign",
"Creative",
"VariableValue",
"VariableName",
)
The MFF long-format columns, in the order of the MFF convention (and mmm-framework).
Every row holds one value: VariableValue of variable VariableName for one
Period and one combination of the key columns.
MFF_KEY_COLUMNS: Final[tuple[str, ...]] = (
"Geography",
"Product",
"Outlet",
"Campaign",
"Creative",
)
The MFF key columns besides Period (each becomes a dim when it has >1 value).
Bases: RequestModel
How an upload becomes a labelled dataset (Meridian CoordToColumns style).
Tabular formats need a mapping; zarr_zip/netcdf are self-describing and may omit
dims/vars (with groups, they rename/select per node by name). A raw hdf5
file (M4) needs hdf5 (:class:Hdf5Mapping) instead of dims/vars. Parsing
happens only inside a sandboxed ingest run.
With the mff preset the long table is first pivoted to one column
per VariableName (plus Period and the key columns); vars and wide then
pick and rename those variables (column = a VariableName value) and dims may
refine the dims the preset derives (column = an MFF column). Without vars and
wide every variable is kept under its own name.
effective_mff: MffOptions | None
The MFF options the ingest run uses (defaults for the mff preset).
Bases: RequestModel
A dim of the dataset, read from one column of a tabular upload.
Bases: RequestModel
A data variable read from one column (dtype is inferred during ingest).
Bases: RequestModel
One variable read from several columns, one per label of dim (a wide table).
{"dim": "channel", "columns": {"tv": "tv_spend", "search": "search_spend"}} gives a
variable along channel (plus the table's other dims) with coords tv and
search. Several wide variables may share a dim; their labels must then agree.
Bases: StrEnum
Bases: StrictModel
The MFF ("model feature format") long-table preset.
One row per (period, geography, product, campaign, outlet, creative, variable) with
the value in value_column. Period becomes the time dim; VariableName values
become data variables. The rest follows mmm-framework's MFF semantics (its
MFFLoader), so both read a file the same way:
GeographyandProductbecome dims when they hold more than one level. A blank label (orNational,Total,All) is the coarser level: a variable given only there (national TV) is spread across the levels - equal weights, or the KPI's share of each level (allocation; mmm-framework's defaults: geographies equally, products by sales). A variable may not mix the two levels.Campaign,OutletandCreativeare breakdowns: rows that differ only there are summed, unless the column is listed insplit(then it becomes a dim).
combine joins several key columns into one dim ({"geo": ["Geography", "Product"]}
gives labels such as US|Watches) - a model with one panel dim can then fit a
geography x product panel. A combined dim is created even when it has a single label,
so {"geo": ["Geography"]} keeps a national table's geo dim.
Bases: RequestModel
Google Meridian's CoordToColumns contract for a wide CSV/Parquet table.
media/media_spend (and reach/frequency/rf_spend) are parallel lists:
the i-th column of each belongs to the i-th entry of media_channels
(rf_channels). geo is omitted for a national model; kpi_type says whether
the KPI is revenue (ROI in currency) or a count (revenue_per_kpi converts it).
wide_vars() -> dict[str, WideVarMapping]
The channel-indexed variables this contract defines (media, spend,
reach, frequency, rf_spend), as :class:WideVarMapping along
channel (rf_channel for reach and frequency).
Business priors and calibration v0.4¶
Business priors, lift tests and prior translation (plan §11 Guided task 3; M3).
Guided users state beliefs in business units instead of model parameters:
- :class:
BusinessPriors- per channel an ROI range[low, high]or a contribution-share range, each held with a statedcredibility(the probability that the truth lies inside the range), plus an optional baseline share; - :class:
LiftTest- the result of an experiment (geo holdout, conversion lift): extra spend on one channel over a window and the incremental KPI it caused, with a standard error or a confidence interval. Lift tests are calibration data: the model treats each one as an extra observation of the channel's incremental effect.
Both attach to a ModelSpec through its optional calibration block (:class:Calibration).
A model plugin uses them only if its manifest declares the matching capability
(Capabilities.calibration); otherwise static validation reports
CALIBRATION_UNSUPPORTED.
Turning business priors into model-space priors needs the data (spend and KPI scale) and the
model's transforms, so it runs in the sandbox as a translate_priors run
(:class:PriorTranslationJob → priors.json = :class:PriorTranslationResult), which
also samples the prior predictive and reports it in business units (ROI and share
distributions, a KPI fan). :func:apply_prior_translation writes the result into a spec.
Pure Pydantic and stdlib (statistics.NormalDist for quantiles): the control plane and
the web app (through OpenAPI) share it.
Bases: StrictModel
Beliefs in business units, per channel (keys are channel labels as in the data), and/or about the baseline. At least one belief is stated: a baseline belief alone is fine (channels without a belief keep the model's default priors).
Bases: StrictModel
What the user believes about one channel: exactly one of roi or
contribution_share, held with credibility.
roi: KPI units gained per spend unit over the fit window ([1.5, 4]= "every euro returns between 1.5 and 4 euros of revenue");contribution_share: the channel's share of the KPI over the fit window.
Bases: StrictModel
A share range (for example of the baseline: KPI that happens without any media).
Bases: StrictModel
One experiment: spend_delta extra spend on channel (optionally in one geo)
from start to end (inclusive) caused kpi_delta incremental KPI.
Give the measurement's uncertainty as a standard error se or a confidence interval
ci at ci_level. A holdout ("turn the channel off") is a negative spend_delta
with (usually) a negative kpi_delta.
Bases: StrictModel
The calibration block of a ModelSpec: beliefs and experiment results the model
should respect. prior_translation_run records which translate_priors run produced
the priors in model.settings.priors (lineage only).
Bases: StrictModel
RunSpec.spec of a translate_priors run.
spec is the resolved ModelSpec document (its data.dataset is the
dataset input). The runner fits nothing: it builds the model, maps the beliefs to
model-space priors and samples draws prior-predictive draws.
Bases: ResponseModel
priors.json of a translate_priors run.
priors are model-space priors in the plugin's settings.priors form (engine-neutral
{distribution, <params>, dims} dicts; per-channel parameters are lists in channels
order), to be written
at :data:PRIORS_POINTER. coverage is the prior mass inside each stated range
(close to the stated credibility when the translation is faithful).
Bases: ResponseModel
The prior predictive in business units: what the model believes before seeing the KPI, so users can check that their beliefs came out as intended.
Bases: ResponseModel
A distribution in business units: mean, sd and named quantiles ("0.05" …).
Bases: ResponseModel
Quantile bands of a series along one dim (a KPI fan): values[i][j] is quantile
quantiles[i] at coords[j]. observed (if any) is the data over coords.
check_business_priors(
priors: BusinessPriors, pointer: str = ""
) -> list[Issue]
Cross-field consistency of business priors (BUSINESS_PRIORS_INVALID issues).
Shares cannot add up to more than the whole KPI: the lower bounds of every channel's share plus the baseline's must stay at or below 1. Upper bounds summing below 1 with a baseline belief leave KPI unexplained, which is a warning.
(mu, sigma) of the LogNormal whose central credibility interval is
[low, high] (the translation of an ROI range). low must be positive.
(mu, sigma) of the logit-normal whose central credibility interval is
[low, high] (the translation of a share range). Needs 0 < low < high < 1.
apply_prior_translation(
document: dict[str, Any],
result: PriorTranslationResult,
*,
business_priors: BusinessPriors | None = None,
run_id: str | None = None,
) -> dict[str, Any]
A copy of the ModelSpec document with result.priors merged into
model.settings.priors (translated keys win) and the calibration block updated
with business_priors and run_id (lineage).
Scorecard and promotion gate v0.4¶
The sanity scorecard and the promotion gate (plan §2 Diagnostic, §11 Guided task 5; M3).
A fitted model version carries a :class:Scorecard: plain-language checks in four
categories, each pass, warn or fail:
============== =========================================================================
category checks (:data:SCORECARD_CHECKS)
============== =========================================================================
convergence chains agree (R-hat), enough effective draws, no divergences
holdout holdout error (MAPE) and interval coverage on the evaluation split; for
choice models, the holdout hit rate against always guessing the most
popular product
plausibility negative or absurd ROI, a channel taking most of the KPI, the baseline share,
parameters the data did not inform (prior-posterior overlap)
data missing values, years of history, spend that never varies
============== =========================================================================
:func:evaluate_check turns a metric value into a :class:ScorecardItem with the check's
message and suggested action; :func:rollup makes the scorecard (overall is the worst
item). Templates pick checks and may tighten thresholds (:class:CheckThresholds).
Promoting a model version to an endpoint passes the promotion gate
(:func:promotion_gate): pass → allowed, warn → needs an administrator override
with a reason (audited), fail → blocked. :func:authorize_promotion applies an override
request to a gate decision.
SCORECARD_CHECKS: Final[dict[str, ScorecardCheck]] = {
c.id: c
for c in (
_c(
"convergence.rhat",
"convergence",
"RHAT",
"Chains agree",
"r_hat_max",
_MAX,
1.01,
1.05,
(
"All chains got the same answer (R-hat {value}).",
"The chains agree, but not fully (R-hat {value}).",
"The chains do not agree (R-hat {value}). The results are not reliable.",
"Increase the tuning steps and the draws, or make the model simpler.",
),
),
_c(
"convergence.ess",
"convergence",
"ESS",
"Enough independent draws",
"ess_bulk_min",
_MIN,
400,
100,
(
"There are enough independent draws ({value}) for stable results.",
"There are only {value} independent draws. The results can change a little from one run to the next.",
"There are only {value} independent draws. The results are not stable.",
"Increase the number of draws.",
),
"int",
),
_c(
"convergence.divergences",
"convergence",
"DIVERGENCES",
"The sampler had no problems",
"divergence_rate",
_MAX,
0.0,
0.01,
(
"The sampler had no problems with the model.",
"The sampler had problems in a small part of the draws ({value}).",
"The sampler had problems in many draws ({value}). The results can be biased.",
"Increase target_accept, or use priors that give more information.",
),
"pct",
),
_c(
"holdout.mape",
"holdout",
"HOLDOUT_MAPE",
"Holdout error",
"mape",
_MAX,
0.1,
0.25,
(
"The forecast error on the holdout periods was {value}.",
"The forecast error on the holdout periods was {value}.",
"The forecast error on the holdout periods was {value}. This error is too large.",
"Look for gaps or one-time events in the last periods of data. Add holidays or controls that explain them.",
),
"pct",
),
_c(
"holdout.coverage_90",
"holdout",
"HOLDOUT_COVERAGE",
"The uncertainty is correct",
"coverage_90",
_BAND,
0.1,
0.25,
(
"{value} of the holdout values were in the 90% range. This is correct.",
"{value} of the holdout values were in the 90% range. The target is 90%.",
"Only {value} of the holdout values were in the 90% range. The uncertainty is not correct.",
"Look for trends or shifts that the model does not show.",
),
"pct",
target=0.9,
),
_c(
"holdout.hit_rate",
"holdout",
"HOLDOUT_HIT_RATE",
"Better than a simple guess",
"accuracy_lift",
_MIN,
0.02,
-0.1,
(
"On the holdout trips, the top choice of the model was correct {value} more often than a guess of the most popular product.",
"On the holdout trips, the top choice of the model was correct about as often as a guess of the most popular product ({value} difference).",
"On the holdout trips, the top choice of the model was correct less often than a guess of the most popular product ({value} difference). Its predicted choices are not reliable.",
"Add the drivers of the choice for each product (price, promotions, displays). Or get more trips for each customer.",
),
"pct",
),
_c(
"plausibility.negative_roi",
"plausibility",
"NEGATIVE_ROI",
"No channel loses money",
"negative_roi_share",
_MAX,
0.1,
0.5,
(
"No channel has a high probability of a negative return.",
"A channel has a {value} probability of a negative return.",
"A channel probably loses money ({value} probability).",
"Do a check of the spend data of the channel. Or give an ROI range in the business priors.",
),
"pct",
),
_c(
"plausibility.roi_absurd",
"plausibility",
"ROI_ABSURD",
"Returns are not too high",
"roi_max",
_MAX,
20.0,
100.0,
(
"The highest return of a channel ({value} for each unit of spend) is in the usual range.",
"A channel returns {value} for each unit of spend. This is higher than usual.",
"A channel returns {value} for each unit of spend. This is too high to be correct.",
"Do a check of the spend units and the gaps. Or give an ROI range in the business priors.",
),
),
_c(
"plausibility.share_outlier",
"plausibility",
"SHARE_OUTLIER",
"No channel is too large",
"max_channel_share",
_MAX,
0.6,
0.9,
(
"No single channel explains most of the KPI.",
"One channel explains {value} of the KPI. This is more than usual.",
"One channel explains {value} of the KPI. This is too high to be correct.",
"Look for a missing control (price, promotions, seasonality).",
),
"pct",
),
_c(
"plausibility.baseline_share",
"plausibility",
"BASELINE_SHARE",
"The baseline is in the usual range",
"baseline_share",
_BAND,
0.4,
0.48,
(
"The baseline (the KPI without media) is {value}. This share is in the usual range.",
"The baseline (the KPI without media) is {value}. This share is unusual.",
"The baseline (the KPI without media) is {value}. This share is too far from the usual range.",
"Give a baseline share in the business priors. Or do a check of the controls.",
),
"pct",
target=0.5,
),
_c(
"plausibility.prior_overlap",
"plausibility",
"PRIOR_OVERLAP",
"The data changed the model",
"prior_overlap_max",
_MAX,
0.9,
0.99,
(
"The data moved each parameter away from its prior.",
"A parameter moved only a small distance from its prior ({value} overlap). The data gives little information about it.",
"A parameter did not move from its prior ({value} overlap). Its value comes from the assumptions, not from the data.",
"Add data where the driver changes, or add a lift test. Or give a business prior that you trust for the driver.",
),
"pct",
),
_c(
"data.missing",
"data",
"DATA_MISSING",
"Few missing values",
"missing_fraction",
_MAX,
0.05,
0.3,
(
"Only {value} of the values are missing.",
"{value} of the values are missing.",
"{value} of the values are missing. This is too many for a reliable model.",
"Fill the gaps in the source data, or remove the series that have gaps.",
),
"pct",
),
_c(
"data.history",
"data",
"DATA_HISTORY",
"Enough history",
"history_years",
_MIN,
2.0,
1.0,
(
"There are {value} years of history. This is enough to find the seasonality.",
"There are only {value} years of history. The seasonal effects are not certain.",
"There are only {value} years of history. This is not sufficient to separate media from seasonality.",
"If you have more history, add it.",
),
"years",
),
_c(
"data.spend_variation",
"data",
"SPEND_VARIATION",
"Spend changes enough",
"spend_cv_min",
_MIN,
0.1,
0.02,
(
"The spend of each channel changes sufficiently to measure its effect.",
"The spend of a channel changes very little ({value}). Its effect is not certain.",
"The spend of a channel almost never changes ({value}). You cannot measure its effect from this data.",
"Add a lift test for the channel, or give its ROI in the business priors.",
),
"pct",
),
)
}
The scorecard check catalog (stable ids; thresholds are the platform defaults).
Bases: ResponseModel
Sanity scorecard: convergence, holdout error and plausibility checks rolled up.
Bases: ResponseModel
ScorecardCheck(
id: str,
category: ScorecardCategory,
code: str,
title: str,
metric: str,
direction: CheckDirection,
warn: float,
fail: float,
pass_plain: str,
warn_plain: str,
fail_plain: str,
suggested_action: str,
value_format: ValueFormat = "num",
target: float | None = None,
)
One check of the catalog. Messages are templates over {value} (formatted per
value_format) and {threshold}.
Bases: StrictModel
Warn and fail thresholds of a check (in the check's direction).
Bases: StrEnum
Bases: ResponseModel
Why the gate is not plainly allowed: a stable code, plain text, and the check.
Bases: ResponseModel
The gate's decision for promoting one model version to an endpoint.
evaluate_check(
check: str | ScorecardCheck,
value: float,
*,
thresholds: CheckThresholds | None = None,
message_expert: str | None = None,
) -> ScorecardItem
The :class:ScorecardItem of one check at value (KeyError for unknown ids,
ValueError for a non-finite value).
rollup(
items: Sequence[ScorecardItem],
skipped: Mapping[str, str] | None = None,
) -> Scorecard
A scorecard whose overall is the worst item; with no item at all it is warn
(nothing was checked).
promotion_gate(
scorecard: Scorecard | None,
*,
fit_mode: str = "evaluate_then_refit",
allow_train_only: bool = False,
partial: bool = False,
schema_breaks: Sequence[str] = (),
schema_change: Literal[
"block", "warn", "allow"
] = "block",
required_checks: Sequence[str] = (),
require_validated: bool = False,
validation_status: str | None = None,
) -> PromotionGateDecision
Decide whether a model version may be promoted (pure).
M3.5: when a governance policy requires validation (require_validated, see
:func:mapi_spec.governance.requires_validation), a version whose validation_status
is not validated is blocked (NOT_VALIDATED): no override promotes it.
- a
partialversion (interrupted fit) is blocked; - a
train_onlyfit is blocked unless the endpoint's policy allows it (Q13:evaluate_then_refitis the default for endpoints, so context and terminal state come from all the data); - breaking input-schema changes (
schema_breaks) followschema_change; - no scorecard needs an override; a
failscorecard blocks, awarnone needs an override, apassone is allowed; - an
evaluate_then_refitversion whose scorecard measured no holdout check (nodata.split, or noevaluate.metrics) was never evaluated and needs an override (NOT_EVALUATED), as does every check ofrequired_checks(a guided template'sscorecard_checks) that the scorecard skipped or lacks.
authorize_promotion(
decision: PromotionGateDecision,
*,
override: bool = False,
reason: str | None = None,
can_override: bool = False,
) -> bool
Apply an override request to a gate decision; return whether it was overridden.
Raises PROMOTION_BLOCKED (blocked: no override possible), OVERRIDE_REQUIRED
(needs an override that was not requested), FORBIDDEN (the caller may not override:
manage on the endpoint, a signed-in admin) or SCHEMA_INVALID (an override
without a reason). The caller audits an override as endpoint.promote_override.
What-if scenarios v0.4¶
Precomputed UI views: small JSON documents the runner writes next to datasets and bundles and the browser reads through signed URLs (the control plane never decodes them).
Every view carries a format tag so readers can refuse versions they do not understand:
===================== ======================== ==========================================
file format content
===================== ======================== ==========================================
preview.json mapi.preview/v1 a long-format sample: dim then var columns
profile.json mapi.profile/v1 per-variable statistics + missingness grid
ui/tree_index.json mapi.tree-index/v1 structure of the bundle DataTree (no arrays)
ui/preprocess.json mapi.preprocess/v1 learned preprocessing state per pipeline step
ui/grids/mapi.response-grid/v1per-draw response curves for what-if sliders
response_curves.json (M3; :class:ResponseGridView)
ui/guided/mapi.forecast/v1a forecast score run's KPI: history, then the
forecast.json forecast fan after the split (M3)
ui/guided/mapi.counterfactual/v1a counterfactual score run's change of the
counterfactual.json KPI, in total and per period (M3)
ui/explore/mapi.model-explorer. quantised posterior rasters of the outputs
fields.jsonfields/v1 that declare a grid (:class:FieldsView)
===================== ======================== ==========================================
Views are produced inside the sandbox from untrusted data: strings are capped and readers must render them as text only.
Bases: _View
ui/grids/response_curves.json: per-draw response curves for instant what-if sliders.
For each thinned posterior draw d, channel c and spend multiplier m,
contribution[d][c][m] is the channel's total contribution to the KPI over the fit
window (periods periods, KPI units) when the channel's spend is scaled by
multipliers[m]; spend[c][m] is that spend (spend units). Multiplier 1 is the
actual spend (current_spend). baseline[d] is the KPI over the window that no
channel explains, so baseline[d] + sum_c contribution[d][c][at 1] is the model's
expected KPI. *_by_geo variants add a trailing geo axis (geos order).
Sliders interpolate per draw (see :func:mapi_spec.whatif.evaluate_scenario), so
uncertainty bands stay joint across channels; "Run exact" submits a real score run.
What-if scenarios, forecasts and counterfactuals for Guided mode (plan §11 tasks 6-7; M3).
Sliders. A :class:WhatIfScenario scales each channel's spend over the fit window.
The browser answers it instantly from the fit's :class:~mapi_spec.views.ResponseGridView
with the interpolation contract below; :func:evaluate_scenario is the reference
implementation (the web app's TypeScript port is tested against the same golden cases):
- a channel without a multiplier keeps
1; multipliers outside the grid ([multipliers[0], multipliers[-1]]) are refused (SCENARIO_OUT_OF_RANGE) - the grid is never extrapolated; - per draw
dand channelc, the contribution at multiplierxis the linear interpolation ofcontribution[d][c]between the two grid points aroundx; spend is interpolated the same way onspend[c]; - per draw,
kpi = baseline + sum_c contributionanddelta = kpi - kpi(all at 1); summaries (mean and quantiles) are taken over draws after summing, so uncertainty stays joint across channels.
Run exact turns the scenario into a counterfactual :class:~mapi_spec.spec.ScoreRequest
(:func:scenario_score_request). Forecasts (:func:forecast_request) and intervention
presets ("channel off", "budget x k"; :func:counterfactual_request) build score requests
the same way, so the server, the CLI and the docs share one implementation
(POST /v1/guided/requests:compile).
Bases: RequestModel
Spend multipliers per channel over the fit window (1 = what was actually spent).
geo restricts the scenario to one geo's grid (needs a per-geo grid).
Bases: ResponseModel
The scenario's outcome over the fit window, in the grid's units.
Bases: ResponseModel
Bases: ResponseModel
Mean and quantiles over posterior draws (quantiles[i] at levels[i]).
Bases: RequestModel
Intervention preset: what if this channel had spent nothing?
Bases: RequestModel
Intervention preset: scale the budget by k (all channels, or the listed ones).
Bases: RootModel[GuidedCompileRequest]
Body of POST /v1/guided/requests:compile: one of whatif, forecast or
counterfactual (by kind); the response is the equivalent ScoreRequest.
evaluate_scenario(
grid: ResponseGridView,
scenario: WhatIfScenario,
*,
levels: Sequence[float] = DEFAULT_QUANTILES,
) -> WhatIfResult
Answer scenario from grid (the reference interpolation; see module doc).
scenario_score_request(
scenario: WhatIfScenario,
model: ModelTarget | str,
*,
spend_var: str = "spend",
channel_dim: str = "channel",
geo_dim: str = "geo",
output: OutputSelector | None = None,
) -> ScoreRequest
"Run exact": the scenario as a counterfactual score over the fit window (one
multiply intervention per changed channel, on the dataset's spend variable).
preset_interventions(
preset: ChannelOff | BudgetScale,
*,
spend_var: str = "spend",
channel_dim: str = "channel",
) -> list[Intervention]
The interventions of a preset (on the dataset's spend variable).
counterfactual_request(
preset: ChannelOff | BudgetScale,
model: ModelTarget | str,
*,
spend_var: str = "spend",
channel_dim: str = "channel",
horizon: Horizon | None = None,
scenario: DatasetRef | str | None = None,
future: FutureCovariates = SUPPLIED,
output: OutputSelector | None = None,
) -> ScoreRequest
A counterfactual over the fit window - or, with a horizon, a forecast with the
preset applied to the future.
forecast_request(
model: ModelTarget | str,
*,
periods: int,
dim: str = "time",
future: FutureCovariates = CARRY_LAST,
scenario: DatasetRef | str | None = None,
output: OutputSelector | None = None,
) -> ScoreRequest
A guided forecast periods ahead along dim. Guided forecasts default to
carrying the last values of future covariates forward; supplied needs scenario.
compile_guided_request(
req: WhatIfCompile
| ForecastCompile
| CounterfactualCompile,
) -> ScoreRequest
The :class:~mapi_spec.spec.ScoreRequest a guided request stands for.
Endpoints and snippets v0.4¶
See Integrations.
Endpoints: stable aliases for promoted model versions (plan §2, §8.2, §11; Q13; M3).
An endpoint endpoint://<project>/<alias> points at one model version at a time.
Integrations (the Python and TypeScript clients, Google Sheets, BI exports) call the
alias, so a new model version can be promoted - or rolled back - without touching them.
- Promotion (
PUT /v1/endpoints/{alias}) passes the promotion gate (:func:mapi_spec.scorecard.promotion_gate): the scorecard, the endpoint's :class:EvaluationPolicy(evaluate_then_refitby default, Q13) and the input-schema compatibility with what callers already send (:func:schema_breaks). - Every promotion and rollback is an :class:
EndpointRevision(the history). - Scoring (
POST /v1/endpoints/{alias}/score) takes an :class:EndpointScoreCall: a score request withoutapiVersion/kind/model, whosedataandscenariomay be dataset references or small inline XrJSON datasets. GET …/schemadescribes the input (:class:EndpointSchema, with an example payload),POST …/validatechecks a call without running it,GET …/snippetsreturns code (:mod:mapi_spec.snippets).
Bases: ResponseModel
A stable alias and the model version it serves now.
Bases: RequestModel
What a model version must satisfy to be promoted to the endpoint (Q13).
fit_mode = evaluate_then_refit (default) requires versions whose production bundle
was refitted on all data after the holdout evaluation; train_only also accepts
exploration fits. schema_change decides what happens when the new version expects
input the endpoint's callers do not send (:func:schema_breaks).
Bases: ResponseModel
The input contract of the promoted version, frozen at promotion: what callers send.
input_schema is the schema at inputs_stage under the spec's canonical names
(data.bind applied: time, y), which inline payloads use; the runner renames
them to the dataset's own names before scoring. horizon_dim is the canonical name of
the dim a forecast extends. conditions_on_actuals: forecasts can carry actuals
(target values for periods after the fit) that the model updates its state with
(:mod:mapi_spec.actuals); the server sets it from the capabilities of the plugin
build that serves the endpoint now, not only from those at fit.
Bases: RequestModel
PUT /v1/endpoints/{alias}: create the alias or point it at another version.
Name the version by id or by model://project/name@vN (exactly one). A version whose
gate says needs_override needs override=true and a reason from a signed-in
project admin (audited); blocked versions cannot be promoted. expected_revision
guards against concurrent promotions (409 CONFLICT when stale).
Bases: ResponseModel
One entry of an endpoint's history (newest first in GET …/history).
Bases: RequestModel
POST /v1/endpoints/{alias}/rollback: point the alias back at the version of an
earlier revision (default: the one before the current). The gate is not re-run - the
version was promoted before - but the rollback is audited with its reason. M3.5: when a
governance policy requires validated models, the earlier version must still be
validated (NOT_VALIDATED otherwise).
Bases: RequestModel
POST /v1/endpoints/{alias}/score (and …/validate): a score request for the
endpoint's current version.
data/scenario are dataset:// references or inline XrJSON datasets (up to
5 MB; :mod:mapi_spec.xrjson). Send Prefer: wait=N (N ≤ 60) to wait: 200 with the
result when the run finishes in time, else 202 with the run.
to_score_request(endpoint: str) -> ScoreRequest
The :class:~mapi_spec.spec.ScoreRequest for endpoint (an
endpoint:// ref). Inline datasets are left out: the server stores them as the
run inputs :data:INPUT_INLINE_DATA / :data:INPUT_INLINE_SCENARIO.
Bases: ResponseModel
GET /v1/endpoints/{alias}/schema: what to send, with a ready example payload.
Bases: ResponseModel
POST /v1/endpoints/{alias}/validate: the static checks of a score call (CoordPlan,
capabilities, inline payload envelope) without running it.
schema_breaks(
old: DataSchema, new: DataSchema
) -> list[str]
Why callers that send data for old would break against new (plain text).
Breaking: a dim that disappears or changes kind; a dim policy that no longer allows
what it did (extend/new_levels → fixed); a new required input variable (not
a target, missing values forbidden); an input variable whose dims or dtype family
change. Dropped variables and loosened policies are compatible.
example_score_call(
snapshot: EndpointSchemaSnapshot, *, periods: int = 8
) -> EndpointScoreCall
An example payload for an endpoint (shown by GET …/schema and the snippets).
Forecasting endpoints get a forecast of periods that carries covariates forward
(no inline data needed); others get a predict call with a one-cell-per-level inline
dataset over the last fitted period, with placeholder zeros for every input variable.
The summary is the model's representation default (mean/hdi_90 for draws,
median/q5/q95 for quantile models, which have no mean), so the example scores.
XrJSON: small labelled datasets inline in JSON requests (plan §3.2; M3 endpoints).
The shape is xr.Dataset.to_dict()::
{"coords": {"time": {"dims": ["time"], "data": ["2026-11-01", ...]}, ...},
"data_vars": {"spend": {"dims": ["geo", "channel", "time"], "data": [[[...]]]}},
"dims": {"time": 8, ...}, # optional
"attrs": {}} # optional
It is for payloads up to :data:MAX_XRJSON_BYTES (one scenario, an endpoint score call),
so curl, TypeScript and Apps Script callers need no upload. The control plane checks only
the envelope here (names, nesting, sizes, element and string caps, ragged lists) and
stores the JSON as a run input; the values are decoded into arrays only inside the runner
sandbox (mapi_sdk wire format), which also checks dtypes and the schema.
Bases: RequestModel
An inline dataset (xr.Dataset.to_dict() shape). Dim lengths must agree across
variables; every dim of a data variable should have a coord (checked in the sandbox
against the model's schema).
Bases: RequestModel
A coord or data variable: data nested len(dims) deep (a scalar for no dims).
Serialized size of an inline payload; raises PAYLOAD_TOO_LARGE over
:data:MAX_XRJSON_BYTES (check before model validation to bound the work).
Code snippets for calling an endpoint (plan §11 Integrations hub; M3).
Pure functions, so the server (GET /v1/endpoints/{alias}/snippets), the CLI and the docs
render identical code. Every snippet scores POST /v1/endpoints/{alias}/score with
Prefer: wait=60 and an example payload, reading the API token from the environment
(MAPI_TOKEN; Apps Script: a script property) - a snippet never contains a secret.
Actuals and future dates. With the endpoint's input table
(:attr:SnippetContext.input_table, EndpointSchema.input_table) the Apps Script reads an
"Input" sheet in that layout - actual rows with the target filled, rows to forecast with it
blank, dates as yyyy-MM-dd - even when the example call sends no data, can create that
sheet, and writes the forecast's conditioning note above the tables in "Result". The other
snippets add a forecast over the table's rows (:attr:SnippetContext.table_call) to fill
with actuals (:mod:mapi_spec.actuals).
Quoting. Values are embedded with each language's own literal syntax, never by string
pasting: Python literals via repr, JavaScript/TypeScript via JSON (with U+2028/U+2029
escaped), shell via single quotes (' → '\''), R via escaped single-quoted strings.
The base URL, project and alias are validated first (:class:SnippetContext), so even a
hostile value cannot break out of its literal; tests parse the generated code back.
Bases: StrEnum
Bases: ResponseModel
Ready-to-run code. code is platform-generated; it embeds the example payload.
Bases: ResponseModel
GET /v1/endpoints/{alias}/snippets (?language= narrows it).
Bases: StrictModel
What a snippet is about. payload is an EndpointScoreCall body (JSON).
input_table (EndpointSchema.input_table) is the endpoint's inline input as a long
table: the Apps Script reads an "Input" sheet with its columns even when payload
sends no data, and, for a forecasting endpoint with example rows, every snippet adds a
forecast over those rows (:attr:table_call) - the call to fill with actuals.
The input table's example rows as mappings (column -> cell).
A forecast over the input table's example rows (None without them): the
periods after the fit with the target and drivers empty - the call a caller fills
with actuals (:mod:mapi_spec.actuals). Empty drivers are carried forward.
render_snippet(
language: SnippetLanguage | str, ctx: SnippetContext
) -> Snippet
One snippet (ValueError for an unknown language).
render_snippets(
ctx: SnippetContext,
languages: Iterable[SnippetLanguage | str]
| None = None,
) -> SnippetSet
Snippets in languages (default: all, in :data:SNIPPET_LANGUAGES order).
Webhooks v0.4¶
Outgoing webhooks: events, envelope, signatures, deliveries and the SSRF policy (plan §8.2 Integrations, §9 "Webhook SSRF"; M3).
Events (:class:WebhookEventType) are delivered as a JSON :class:WebhookEnvelope::
{"id": "evt_…", "type": "run.succeeded", "created": "2026-10-15T09:30:00Z",
"tenant": "acme", "project": "growth", "api_version": "v1", "data": {...}}
Signatures. Every delivery carries Mapi-Signature: t=<unix seconds>,v1=<hex> where
v1 is HMAC-SHA256 under the webhook secret of "<t>.<raw body>". While a secret is
being rotated the header carries one v1 per active secret. Receivers verify with
:func:verify_signature and refuse timestamps more than :data:SIGNATURE_TOLERANCE_S
(300 s) away from their clock, which bounds replays.
SSRF. The server delivers only under :data:SSRF_POLICY: https only, allowed
ports, no credentials in URLs, blocked internal host names; it resolves the host itself,
refuses if any address is private, loopback, link-local, metadata, CGNAT, ULA,
multicast or otherwise non-global (:func:ip_is_blocked, including IPv4 embedded in IPv6),
connects to the pinned address it checked (SNI and Host keep the name), follows no
redirects, gives up after 5 s and reads at most 256 KiB of the response.
Bases: RequestModel
POST /v1/webhooks: the secret is generated by the server and shown once.
The server applies the SSRF policy to url (:func:check_webhook_url, then
:func:check_resolved_addresses at every delivery): 422 WEBHOOK_URL_BLOCKED.
Bases: RequestModel
PATCH /v1/webhooks/{webhook_id} (re-activating resets the failure count).
Bases: ResponseModel
A webhook as listed (never its secret; secret_hint shows its last 4 characters).
Bases: ResponseModel
201 of POST /v1/webhooks and 200 of …:rotate-secret: the only time the
secret is returned. After a rotation the previous secret keeps signing for 24 h.
Bases: ResponseModel
The JSON body of every delivery. data depends on type:
run.*:{run_id, kind, state, failure_class?, code?, model_version_id?};model.version.ready:{model_id, model_version_id, ref, scorecard_overall};endpoint.promoted:{alias, ref, revision, model_version_id, overridden};score.completed:{run_id, endpoint?, model_version_id, outputs: [names]};tenant.deletion.completed:{deletion_id, requested_at, completed_at}.
No envelope carries data values, signed URLs or secrets: fetch results with the API.
Bases: ResponseModel
One delivery attempt chain of one event to one webhook (attempt counts tries).
error is platform-authored; response_excerpt is the first bytes of the receiver's
answer (untrusted text: render as text only).
SsrfPolicy(
schemes: tuple[str, ...] = ("https",),
ports: tuple[int, ...] = (443, 8443),
blocked_networks: tuple[
IPv4Network | IPv6Network, ...
] = _BLOCKED_NETS,
blocked_hosts: frozenset[str] = frozenset(
{
"localhost",
"metadata",
"metadata.google.internal",
"instance-data",
}
),
blocked_suffixes: tuple[str, ...] = (
".localhost",
".local",
".internal",
".intranet",
".lan",
".home.arpa",
".corp",
),
resolve_then_check: bool = True,
pin_resolved_ip: bool = True,
follow_redirects: bool = False,
timeout_s: float = 5.0,
max_response_bytes: int = 256 * 1024,
max_url_length: int = 2048,
user_agent: str = "mapi-webhooks/1",
allow_ip_literals: bool = False,
)
How the server delivers webhooks (data, so docs and tests read the same values).
The hex HMAC-SHA256 of "<timestamp>.<body>" under secret.
t=<unix>,v1=<hex>[,v1=<hex>…] - one v1 per active secret (rotation).
(timestamp, [v1 signatures]); unknown schemes are ignored. ValueError if
malformed (no or several t, a bad v1, no v1 at all, over 4 KB).
check_signature(
body: bytes,
header: str | None,
secrets_: str | Iterable[str],
*,
now: int | None = None,
tolerance: int = SIGNATURE_TOLERANCE_S,
) -> int
Verify a delivery; return its timestamp or raise WEBHOOK_SIGNATURE_INVALID.
Accepts if any v1 matches any of secrets_ (constant-time comparison) and the
timestamp is within tolerance seconds of now (either direction).
verify_signature(
body: bytes,
header: str | None,
secrets_: str | Iterable[str],
*,
now: int | None = None,
tolerance: int = SIGNATURE_TOLERANCE_S,
) -> bool
:func:check_signature as a boolean (for receivers).
next_retry_delay(
attempt: int,
response_code: int | None = None,
*,
jitter: float = 0.0,
) -> timedelta | None
The delay before the next try after attempt failed, or None to give up.
2xx is success (no retry); 410 Gone never retries; jitter in [0, 1) adds up to
10% (callers pass a random number; tests pass a fixed one).
ip_is_blocked(
address: str | IPv4Address | IPv6Address,
policy: SsrfPolicy = SSRF_POLICY,
) -> bool
Is address unsafe to connect to? Covers every non-global range and IPv4 embedded
in IPv6 (mapped ::ffff:a.b.c.d, NAT64 64:ff9b::/96, 6to4 2002::/16).
check_webhook_url(
url: str, policy: SsrfPolicy = SSRF_POLICY
) -> str
Static checks of a webhook URL (before any DNS). Returns the lower-cased host.
Raises WEBHOOK_URL_BLOCKED for: another scheme, credentials, a fragment, a port
outside policy.ports, an IP literal, a blocked host name or suffix, a single-label
host, or characters outside a plain DNS name.
check_resolved_addresses(
addresses: Sequence[str],
policy: SsrfPolicy = SSRF_POLICY,
) -> str
After resolution: every address must be allowed; return the one to pin (the first).
Raises WEBHOOK_URL_BLOCKED when there is none or any is blocked (a host with one
private and one public address is refused: DNS answers can rotate).
Plot sharing v0.4¶
Saved plots, static rendering and embeds (plan §8.2 Plots, §10.3 items 4-5; M3).
- Saved plots (
POST /v1/plots) keep a plot under a name: either a reference to a plot a model version rendered (a platform recipe or a plugin plot) or a user-authored :class:~mapi_spec.plots.PlotSpec(sanitized on save and on every read). - Static rendering (
POST /v1/plots:render) turns a plot into PNG, SVG or PDF in the sandbox (vl-convert for Vega-Lite, kaleido for Plotly): arenderrun. - Embeds (
POST /v1/plots/{plot_id}/embeds) mint a tokenised, read-only, expiring link for BI tools, Slack and Notion. Plan §10.3 item 4: shared views render platform recipes only - never plugin- or user-authored specs - because a sandboxed iframe can still navigate itself; :func:embeddabledecides. The publicGET /v1/embeds/{token}returns an :class:EmbedViewwith every dataset inlined (no URLs).
Embed tokens are mpe_<40 base32 characters> (200 random bits); the server stores only
an HMAC (like API tokens) and shows the URL once.
Bases: RequestModel
POST /v1/plots: exactly one of source (a model version's plot) or spec
(a user-authored plot, sanitized first; never embeddable).
Bases: ResponseModel
A saved plot. spec is the sanitized PlotSpec (on GET /v1/plots/{plot_id};
omitted in lists). embeddable is :func:embeddable of this plot.
Bases: RequestModel
A plot a model version rendered (GET /v1/models/{id}/versions/{v}/plots).
Bases: RequestModel
POST /v1/plots:render: exactly one of plot (an inline spec, sanitized),
saved_plot_id or source. width/height are CSS pixels before scale;
the raster (width x height x scale^2, with the plot's own size where none is given)
holds at most :data:MAX_RENDER_PIXELS. Send Prefer: wait=N to get the file links in
the response.
Bases: RequestModel
POST /v1/plots/{plot_id}/embeds. allowed_origins (https origins) become the
embed page's frame-ancestors; empty allows any site to frame it.
Bases: ResponseModel
An embed link as listed (never its token).
Bases: ResponseModel
201 of POST …/embeds: the only time the URL (with its token) is returned.
Bases: ResponseModel
GET /v1/embeds/{token} (public, rate-limited, cacheable until expires_at):
a platform-recipe plot with every dataset inlined. Expired or revoked tokens answer
410 LINK_EXPIRED; unknown ones 404.
Plan §10.3 item 4: only plots a model version rendered from a platform recipe.
core.ppc_fan@1.0.0 → True; plugin or unknown namespaces → False.
Trust lifecycle v0.4¶
Trust lifecycle: self-service sign-up, verification, tenant deletion and GDPR export (plan §9 "Compute abuse", "Data lifecycle", §14 M3 and cut lines A/B; M3).
- Sign-up (
POST /auth/signup): e-mail, name and accepted terms. The answer is the same whether or not the address is new (no account enumeration). A disposable-e-mail policy hook (:class:DisposableEmailPolicy) refuses throwaway domains. - E-mail verification (
POST /auth/verify-email) with a single-use token that expires after :data:EMAIL_VERIFICATION_TTLcreates apublic-tier tenant with adefaultproject, the signer as tenant admin. - Cut line A (M3): public tenants run declarative specs and built-in plugins only -
untrusted data, trusted code. Cut line B (M5) opens code upload to public tenants, and
only after identity verification (:func:
uploaded_code_allowed). - Tenant deletion (
POST /v1/tenants/{id}/deletion) is scheduled with a grace period (:data:TENANT_DELETION_GRACE, the tenant is suspended and can cancel), then purges every store and reports progress per store (:class:TenantDeletion). - GDPR export (
POST /v1/tenants/{id}/exports) packages everything the tenant stored into one archive with a short-lived download link (:class:GdprExport).
Bases: RequestModel
POST /auth/signup (public, rate-limited). organisation names the tenant.
Bases: ResponseModel
202 of POST /auth/signup: identical for new and known addresses.
Bases: Protocol
The hook the server consults at sign-up (a static list, a vendor API, ...).
Refuses a domain on the list and any of its subdomains.
Bases: RequestModel
POST /auth/verify-email: the token from the e-mail link (single use).
Bases: ResponseModel
The verified address and the public-tier tenant it now administers; sign in next.
Bases: ResponseModel
GET /v1/tenants/{tenant_id}/verification.
Bases: StrEnum
Bases: RequestModel
PUT /v1/admin/tenants/{tenant_id}/verification (operators; audited).
Bases: RequestModel
POST /v1/tenants/{tenant_id}/deletion: confirm must repeat the tenant id.
Bases: ResponseModel
A tenant deletion and its progress per store.
Bases: ResponseModel
Bases: RequestModel
POST /v1/tenants/{tenant_id}/exports: what to include (default: everything).
Bases: ResponseModel
An export job; download_url is a short-lived signed link once ready.
check_signup(
req: SignupRequest,
policy: DisposableEmailPolicy | None = None,
) -> str
Refuse a sign-up early; return the normalised e-mail domain.
TERMS_NOT_ACCEPTED without accepted (current) terms; EMAIL_NOT_ALLOWED for a
malformed or disposable address.
The normalised (lower-case, IDNA/punycode) domain of an address (ValueError).
uploaded_code_allowed(
tier: TrustTier,
*,
identity_verified: bool,
public_code_enabled: bool = CUT_LINE_B_OPEN,
) -> tuple[bool, str]
May this tenant upload plugin code (and plain-language why)?
internal/self-hosted-trusted: yes;verified(allow-listed, M4): only once identity is verified;public: never before cut line B; afterwards only with identity verification.
next_deletion_state(
state: DeletionState, event: DeletionEvent
) -> DeletionState
Apply event (CONFLICT if the transition is not allowed).
About cards and the chooser v0.5¶
Plain-language "About" cards and the "Help me choose" chooser (M3.5).
Every item a user picks from a catalog - models, preprocessing steps and splits, guided
templates and specs, workflow templates, workflow node kinds and node implementations,
governance checks, datasets - carries an :class:About: what it is in one or two sentences
without jargon, when to use it and when not to, the data it needs, what it assumes, what you
get out, how hard it is, how long it usually takes, a tiny example and related items. Expert
detail (maths, references) lives apart in expert_notes and references.
- :func:
about_issuesis the completeness rule the catalog and the contract kit apply (mapi_spec.catalog.catalog_about_issues,mapi_sdk.testing.check_about); - :class:
DatasetAboutcombines a user's description of a dataset with a summary generated from its schema and coordinate summary (:func:describe_dataset, pure); - the chooser asks :data:
CHOOSER_QUESTIONSand ranks candidates by their :attr:About.fitstags (:func:recommend, pure), behindGET /v1/catalog/chooserandGET /v1/catalog/recommendations.
Pure Pydantic and stdlib (no numpy): the control plane, the web app and the docs share it.
ABOUT_JARGON: Final[frozenset[str]] = frozenset(
{
"autoregressive",
"autocorrelation",
"covariance",
"eigenvalue",
"heteroskedastic",
"heteroskedasticity",
"heteroscedastic",
"homoskedastic",
"hyperparameter",
"hyperprior",
"posterior",
"stationarity",
"stochastic",
"cointegration",
"cointegrated",
"multicollinearity",
"likelihood",
"regularisation",
"regularization",
"orthogonal",
"asymptotic",
"iid",
"mcmc",
"nuts",
"arima",
"sarimax",
"varmax",
"vecm",
}
)
Words that must not appear in a summary_plain (they may in the other fields, which
explain them, and in expert_notes).
CHOOSER_QUESTIONS: Final[tuple[ChooserQuestion, ...]] = (
_q(
"goal",
"What do you want to do?",
[
(
"explain_marketing",
"Measure what my marketing adds",
),
(
"plan_budget",
"Plan or split a marketing budget",
),
(
"measure_impact",
"Measure the effect of a campaign, a launch or a test",
),
("forecast", "Forecast the future"),
(
"explain_drivers",
"Understand what causes changes in a number",
),
(
"predict_values",
"Predict a value for each store, product or customer",
),
(
"understand_choices",
"Understand which product people choose",
),
(
"find_regimes",
"Find periods that behave differently",
),
(
"compare_series",
"See how several series move together",
),
("check_data", "Examine or prepare my data"),
(
"validate_model",
"Make sure that I can trust a model",
),
(
"map_fields",
"Make a map or an image from measurements, with its uncertainty",
),
(
"fit_physics",
"Fit a physical model to measurements",
),
(
"classify",
"Put items or images into classes",
),
],
weight=3,
multiple=True,
),
_q(
"data_shape",
"What does your data look like?",
[
("one_series", "One number over time"),
(
"several_series",
"A few related numbers over time",
),
(
"panel",
"The same numbers for many markets, stores or products",
),
(
"choices",
"Purchases or choices by customers",
),
(
"table",
"A table of items without a time order",
),
(
"grid",
"A map, an image or values on a regular grid",
),
(
"images",
"Images (pictures of the same size)",
),
],
weight=2,
multiple=True,
),
_q(
"frequency",
"How often do you record the data?",
[
("daily", "Daily"),
("weekly", "Weekly"),
("monthly", "Monthly or less often"),
("not_time", "It is not over time"),
],
multiple=True,
),
_q(
"history",
"How much history do you have?",
[
("short", "Less than a year"),
("medium", "One to three years"),
("long", "More than three years"),
],
),
_q(
"drivers",
"Do you have other columns that could explain the result?",
[
(
"yes",
"Yes (prices, promotions, weather, spend)",
),
("no", "No, only the number"),
],
help="Columns that you also know for the future (planned prices, booked spend) make forecasts better.",
),
_q(
"uncertainty",
"Do you need a range (best and worst case), not only one number?",
[
("essential", "Yes, a range is essential"),
("nice", "It would be good"),
("no", "One number is enough"),
],
),
_q(
"experience",
"How much do you know about statistics?",
[
("beginner", "New to it"),
("intermediate", "I know the basics"),
("expert", "I am an expert"),
],
),
_q(
"speed",
"How quickly do you need results?",
[
("seconds", "Within seconds"),
("minutes", "Within minutes"),
("can_wait", "I can wait"),
],
),
)
The chooser's questions (stable ids; :attr:About.fits keys and values refer to them).
Bases: StrictModel
The plain-language card of one catalog item.
Every field except the expert ones is required, so an About that validates is
structurally complete; :func:about_issues adds the editorial rules (one or two
sentences, no jargon in the summary, known chooser tags, related ids that exist).
Bases: StrEnum
What an About describes (chooser results and the catalog say which).
Bases: StrictModel
A source for the expert detail (a paper, a book chapter, library documentation).
Bases: StrEnum
How much statistical background an item needs to use well.
Bases: StrEnum
Rough wall-clock time on typical data (S resource class).
Bases: ResponseModel
The About of a dataset: the owner's description plus a generated summary.
summary_generated and highlights come from the version's schema and coordinate
summary (:func:describe_dataset); warnings_plain from its ingest issues and profile.
Bases: RequestModel
PATCH /v1/datasets/{dataset_id}/about: the owner's own description.
Bases: ResponseModel
One plain-language question of the chooser (multiple: several answers allowed).
Bases: ResponseModel
Bases: ResponseModel
GET /v1/catalog/chooser: the questions, in the order to ask them.
Bases: RequestModel
One answered question (values: one choice, or several for multiple).
Bases: ResponseModel
One ranked candidate with the reasons in plain words.
Bases: ResponseModel
GET /v1/catalog/recommendations: candidates ranked best first.
about_issues(
about: About | None,
*,
known_ids: Iterable[str] | None = None,
) -> list[str]
Plain-language problems that make an About incomplete (empty list: complete).
None is one problem ("no About"). known_ids, when given, are the catalog ids
related may point at.
describe_dataset(
schema: DataSchema | None,
summary: DataSummary | None,
*,
description: str | None = None,
warnings: Sequence[str] = (),
) -> DatasetAbout
A plain-language summary of a dataset version from metadata alone (pure; no arrays).
Example: Weekly data from 2023-01-01 to 2024-12-29 (104 weeks) for 10 markets ....
chooser_answers_from_query(
query: Mapping[str, Sequence[str]],
) -> list[ChooserAnswer]
Answers from the recommendations query string (?goal=forecast&data_shape=panel;
repeat a key or separate with commas for several values). Unknown keys other than
kind and limit are CHOOSER_ANSWER_INVALID.
recommend(
answers: Sequence[ChooserAnswer],
candidates: Iterable[
tuple[AboutKind | str, str, About]
],
*,
limit: int = 10,
include_about: bool = False,
fallback: Iterable[tuple[AboutKind | str, str, About]]
| None = None,
) -> ChooserResult
Rank candidates ((kind, id, About)) against answers (pure, deterministic).
Each answered question an item declares in :attr:About.fits adds the question's weight
when an answer matches (a reason) or subtracts it when none does (a caution);
undeclared questions are neutral. experience below the item's difficulty and
speed below its typical runtime cost one point per level. Scores are normalised to
[0, 1]; ties break by the number of answers an item matches explicitly (an item made
for every answer before one that is silent on some), then difficulty, then id.
When goal is answered, an item made for other goals is never recommended (items
whose goal does not match are dropped). When no candidate matches the goal,
fallback (other kinds: the building blocks behind the starting points) is ranked
instead.
About cards of the built-in catalog (M3.5): steps, splits, models and guided templates.
The text is written for people who do not do statistics for a living: the summary says what
the item does for them, the lists say when it fits, what it needs and what comes out, and
expert_notes keeps the maths apart. Workflow nodes and governance checks keep their
cards next to their registries (:mod:mapi_spec.workflow_nodes,
:mod:mapi_spec.governance); :func:builtin_about looks up any of them by id.
Built-in steps and models take their card from here when their class does not set one
(mapi_sdk.steps.Step.about, mapi_sdk.models.CustomModel.about). Models that are built
in other v0.5 components (mmm-framework and statsmodels) are listed too, so their plugins and
the chooser agree on the words: :data:PLANNED_MODEL_IDS.
STEP_ABOUTS: Final[dict[str, About]] = {
"core.resample": _a(
"Weekly or monthly totals",
"Adds the daily (or hourly) numbers together into one number for each week or month. Each row then covers the same length of time.",
[
"Your data is daily, but you plan and report by week or month",
"A model needs weekly data",
],
[
"Your data already has one row for each week or month",
"You cannot add the numbers together, and an average hides the important information",
],
["A date column with a regular interval"],
[
"You can add spend and sales together. Prices and rates get an average (select this for each column)",
"When the step needs full periods, it removes an incomplete first or last period",
],
[
"The same columns with one row for each week or month"
],
_B,
_NOW,
"If the daily TV spend is 100 every day, the week that ends on Sunday gets 700.",
related=["core.count_events", "core.calendar"],
expert="Group-by on the time coordinate at the target frequency (pandas offset alias). `how` for each variable (sum, mean, last). `full_periods_only` removes partial bins.",
fits=_PREP,
),
"core.count_events": _a(
"Count events per period",
"Changes a list of individual events, such as orders or sign-ups, into counts for each day, week or month.",
[
"Your dataset has one row for each event and you need a time series"
],
[
"Your data is already a series of totals for each period"
],
["A column with the time of each event"],
[
"Each row is one event. The step ignores events outside the dates that you select"
],
[
"A count for each period (and for each group if you select one), and optional sums"
],
_B,
_NOW,
"Three orders on Monday and one order on Tuesday become 3 and 1.",
related=["core.resample"],
fits=_PREP,
),
"core.filter_coords": _a(
"Keep some markets, products or dates",
"Keeps only the markets, products or date range that you select. It removes the rest before you fit the model.",
[
"You do not want to model some markets or products",
"The early history is not reliable and you want to remove it",
],
[
"You want the model to learn from all the data",
"You want to test on markets that the model did not see (use a split instead)",
],
["A dimension such as market, product or date"],
[
"The model and the steps after this step do not need the removed data"
],
[
"The same data with fewer markets, products or dates"
],
_B,
_NOW,
"Keep only the markets DE and FR, or only the weeks from 2023 and after.",
related=["core.leave_group_out"],
fits=_PREP,
),
"core.to_long": _a(
"One row per observation",
"Changes a table where the rows of each group are side by side into one row for each observation. It removes the empty padding.",
[
"A model works on individual observations, such as shopping trips",
"Groups have very different numbers of rows",
],
[
"All groups have the same number of rows and the model needs a grid"
],
[
"A group dimension (customer) and a position dimension (trip number)"
],
["Missing values or a mask mark the padding cells"],
[
"The same values with one row for each observation and a group index"
],
_I,
_NOW,
"Household 1 with 3 trips and household 2 with 1 trip become 4 rows.",
related=["core.to_padded"],
fits=_PREP,
),
"core.to_padded": _a(
"Put groups side by side",
"Puts the rows of each group side by side in a grid. It fills short groups with gaps and marks the gaps as missing.",
[
"A model needs a grid of groups by position, such as households by trip"
],
["Your data is already a grid"],
["A group column and an order in each group"],
[
"The gaps have a mark, so the model can ignore them"
],
[
"A grid of groups by position, and a mask of the real cells"
],
_I,
_NOW,
"Households with 3 trips and 1 trip become a 2 by 3 grid with two empty cells.",
related=["core.to_long"],
fits=_PREP,
),
"core.stack": _a(
"Combine dimensions",
"Combines two or more dimensions, such as market and product, into one list of cells.",
[
"A model works on a flat list of cells, not on a grid"
],
["The model can use the grid directly"],
["Two or more dimensions to combine"],
[
"The step can remove empty combinations. It keeps a record of them, so you can put them back"
],
["One combined dimension"],
_I,
_NOW,
"3 markets by 4 products become 12 market-product cells.",
related=["core.unstack"],
fits=_PREP,
),
"core.unstack": _a(
"Split a combined dimension",
"Divides a combined dimension back into its parts. For example, market-product cells become markets and products again.",
[
"You stacked dimensions in an earlier step and you need the grid again"
],
["No step stacked the dimensions"],
["A dimension that a stack step made"],
["The parts of each combined label are known"],
["The original dimensions"],
_I,
_NOW,
"12 market-product cells become 3 markets by 4 products.",
related=["core.stack"],
fits=_PREP,
),
"core.formula": _a(
"New column from a formula",
"Makes a new column from a simple formula of existing columns, such as price times units.",
[
"You need a derived number, for example revenue = price x units",
"A ratio or a difference of two columns explains the result better",
],
[
"A built-in step already does it (for example scale, log or lag)"
],
["The columns that the formula uses"],
[
"The formula uses only the permitted operations (arithmetic and some functions). No code runs"
],
["One new column"],
_I,
_NOW,
"revenue = price * units adds a revenue column.",
expert="The step compiles a Lark grammar to an allow-listed AST of xarray operations and evaluates it in the sandbox. This is a usability feature, not a security boundary.",
fits=_PREP,
),
"core.adstock": _a(
"Carry-over of advertising",
"Spreads the advertising effect of each period over the next periods. People remember an ad for some time after they see it.",
[
"You model media with a simple (non-Bayesian) model and you need carry-over",
"You want to see how a fixed carry-over rate changes spend series",
],
[
"The model already calculates carry-over itself (the Bayesian MMM does this)"
],
["Media spend or exposure over time"],
[
"You set the carry-over rate. The model does not learn it from the data"
],
[
"Media columns that include the remaining effect of past periods"
],
_I,
_NOW,
"With a 50% weekly carry-over, 100 spent this week counts as 100, then 50, then 25.",
related=["core.saturation", "mmm.bayesian"],
expert="Geometric (alpha), delayed (theta) or Weibull kernels up to l_max lags, with optional normalisation to a sum of 1. The step applies them to each group along time.",
fits={"goal": ["explain_marketing", "check_data"]},
),
"core.saturation": _a(
"Diminishing returns",
"Bends spend so that each extra unit of money adds a little less than the unit before. This is what occurs when a channel is saturated.",
[
"You model media with a simple model and you need diminishing returns"
],
[
"The model learns saturation itself (the Bayesian MMM does this)"
],
["Media spend or exposure"],
[
"You set the shape and its settings. The model does not learn them from the data"
],
["Media columns on a curved scale"],
_I,
_NOW,
"With half-saturation at 1000, a spend of 1000 gives half of the maximum effect.",
related=["core.adstock", "mmm.bayesian"],
expert="Logistic (lam), Hill (slope, half) or Michaelis-Menten curves, applied element by element.",
fits={"goal": ["explain_marketing", "check_data"]},
),
"core.fx_convert": _a(
"Convert currencies",
"Converts money columns from local currencies into one currency. It uses exchange rates for each day or week.",
["Markets report in different currencies"],
["All values are already in one currency"],
[
"Money columns and a table of exchange rates over time"
],
[
"The exchange-rate table covers all dates in the data"
],
["Money columns in the currency that you select"],
_B,
_NOW,
"1000 GBP on a day when 1 GBP = 1.17 EUR becomes 1170 EUR.",
related=["core.resample"],
fits=_PREP,
),
"core.mean_scale": _a(
"Put series on a common scale",
"Divides each series by its average, so that you can compare big and small markets on the same scale.",
[
"Markets or channels are very different in size",
"A model works better when each series has an average of 1",
],
["The model scales the data itself"],
["Numeric columns"],
[
"The step learns the average from the training data only and uses it again for new data"
],
[
"The same columns with an average of 1 (the server converts the results back automatically)"
],
_B,
_NOW,
"A market that sells 5000 a week on average becomes 1.0. A week with 6000 becomes 1.2.",
related=["core.standardize", "core.max_abs_scale"],
fits=_PREP,
),
"core.max_abs_scale": _a(
"Scale to between -1 and 1",
"Divides each series by its largest absolute value, so all values are between -1 and 1.",
["A model needs values in a fixed range"],
[
"Your data has extreme outliers (they set the scale)"
],
["Numeric columns"],
[
"The step learns the largest value from the training data only"
],
["The same columns between -1 and 1"],
_B,
_NOW,
"Values 50, 100 and -20 become 0.5, 1.0 and -0.2.",
related=["core.mean_scale", "core.clip"],
fits=_PREP,
),
"core.standardize": _a(
"Centre and scale",
"Moves each series so that its average is zero, and stretches it so that its typical spread is one.",
[
"You compare the size of effects across inputs that have different units"
],
[
"Zero must keep its meaning (a spend of zero must stay zero)"
],
["Numeric columns"],
[
"The step learns the average and the spread from the training data only"
],
["The same columns with average 0 and spread 1"],
_B,
_NOW,
"If temperatures have an average of 15 and a spread of 5, 20 degrees becomes 1.0.",
related=["core.mean_scale"],
fits=_PREP,
),
"core.log1p": _a(
"Compress large values",
"Takes the logarithm of one plus each value. This makes very large numbers smaller and changes growth rates into steady steps.",
[
"Values cover several orders of magnitude",
"Effects work in percentages",
],
["Values can be negative"],
["Numeric columns with values of zero or more"],
[
"The server converts the results back. Averages on the original scale are approximate"
],
["The same columns on a log scale"],
_B,
_NOW,
"Sales of 9, 99 and 999 become about 2.3, 4.6 and 6.9.",
related=["core.logit"],
fits=_PREP,
),
"core.logit": _a(
"Unbounded scale for shares",
"Changes shares between 0 and 1, such as conversion rates, into numbers without limits. Then a model cannot predict impossible values.",
["You model a rate or a share"],
["Values are not between 0 and 1"],
[
"Columns with shares that are strictly between 0 and 1"
],
[
"The step moves exact values of 0 and 1 a small distance inwards"
],
["The same columns on an unbounded scale"],
_I,
_NOW,
"A 50% share becomes 0. A 90% share becomes about 2.2.",
related=["core.log1p"],
fits=_PREP,
),
"core.impute": _a(
"Fill gaps",
"Fills missing values with a sensible substitute, such as the average, zero or the previous value.",
[
"Some values are missing and the model cannot use data with gaps"
],
[
"Many values are missing (correct the source instead)",
"A missing value has a meaning (add a missing-value flag)",
],
["The columns with gaps"],
[
"The step learns the fill values from the training data only"
],
["The same columns without gaps"],
_B,
_NOW,
"The step fills a missing week of spend in a market with the average of that market.",
related=[
"core.missing_indicator",
"core.replace_sentinel",
],
fits=_PREP,
),
"core.replace_sentinel": _a(
"Placeholder numbers to missing",
"Marks placeholder numbers, such as -99 or 9999, as missing. Some systems use these numbers for 'unknown'.",
[
"Your source uses codes such as -99 for missing values"
],
["All numbers are real measurements"],
["Columns with placeholder codes"],
["The codes are never real values"],
[
"The same columns with gaps in place of the codes"
],
_B,
_NOW,
"A temperature of -99 becomes a missing value.",
related=["core.impute"],
fits=_PREP,
),
"core.clip": _a(
"Cap extreme values",
"Changes values below or above the limits that you select to the nearest limit. This controls outliers.",
[
"Some extreme values have too much effect on the model",
"Values outside a range are errors",
],
["The extreme values are real and important"],
[
"Numeric columns and a lower limit, an upper limit or both"
],
[
"Values beyond the limits do not have more information"
],
["The same columns in the limits"],
_B,
_NOW,
"With an upper limit of 100, a value of 250 becomes 100.",
related=["core.max_abs_scale"],
fits=_PREP,
),
"core.fourier": _a(
"Repeating seasonal patterns",
"Adds smooth wave-shaped columns. With these columns, a model can learn a repeating pattern, such as a pattern for each year or each week.",
[
"Your result increases and decreases with the season",
"You want seasonality without one column for each month",
],
["You have less than one full season of history"],
["A regular date column"],
[
"The pattern repeats with the same length in each cycle"
],
[
"Pairs of wave columns (more pairs give a more detailed shape)"
],
_I,
_NOW,
"Three yearly wave pairs on weekly data show a summer peak and a winter dip.",
related=["core.calendar", "core.holidays"],
expert="sin/cos(2 pi k t / period), k = 1..order. The origin and the period are stored state, so forecasts continue the same phase.",
fits=_PREP,
),
"core.trend": _a(
"Steady drift over time",
"Adds a column that counts up over time, so a model can show a steady increase or decrease.",
["Your result increases or decreases steadily"],
[
"The trend changes direction frequently (use a flexible trend model)"
],
["A regular date column"],
[
"The drift continues at the same rate into the future"
],
["A trend column (optionally curved)"],
_B,
_NOW,
"Week 1 gets 0, week 52 gets 1, and week 104 gets 2.",
related=["core.fourier", "ts.hsgp_trend"],
fits=_PREP,
),
"core.calendar": _a(
"Calendar facts",
"Adds calendar facts, such as the month, the week of the year, or if a day is on a weekend.",
[
"Behaviour changes with the day of the week or the month"
],
["Your data is not over time"],
["A date column"],
["The same calendar effect occurs every year"],
["One column for each calendar feature"],
_B,
_NOW,
"2024-06-01 gets month 6 and weekend = yes.",
related=["core.holidays", "core.fourier"],
fits=_PREP,
),
"core.holidays": _a(
"Holiday flags",
"Marks the periods that contain public holidays. It uses a holiday table that you supply.",
["Holidays change your sales"],
["Holidays do not change your result"],
[
"A date column and a holiday table (for each country or market)"
],
[
"The holiday table covers the full period, and also the forecast"
],
["A holiday flag or a count for each period"],
_B,
_NOW,
"The step flags the week that contains 25 December in each market that has this holiday.",
related=["core.calendar"],
fits=_PREP,
),
"core.lag": _a(
"Earlier values as columns",
"Adds the values from one or more earlier periods as extra columns, such as the sales of last week.",
[
"The value of the last period helps to predict this period"
],
[
"You forecast many periods ahead and the lagged value is not known"
],
["A time dimension and the columns to lag"],
[
"Earlier rows exist. The first rows get a fill value"
],
["One new column for each lag"],
_I,
_NOW,
"A one-week lag of sales puts the 120 of last week next to the 130 of this week.",
related=["core.lag_tensor"],
fits=_PREP,
),
"core.lag_tensor": _a(
"Recent history per series",
"Keeps the last periods of each series side by side. Models use them to calculate carry-over effects themselves.",
[
"A model calculates its carry-over from a window of past spend"
],
[
"The model takes plain series and uses the history internally"
],
[
"A time dimension and the columns that need history"
],
[
"The window is sufficiently long for the effect to decrease to zero"
],
[
"A new dimension with the last periods of each series"
],
_X,
_NOW,
"With a 6-week window, each week has the spend of the 6 weeks before it.",
related=["core.lag", "core.adstock"],
fits=_PREP,
),
"core.one_hot": _a(
"Categories to yes/no columns",
"Changes a category, such as region, into one yes/no column for each value. Then models that need numbers can use it.",
["A model needs numbers but you have categories"],
[
"A category has hundreds of values (give them numbers instead)"
],
["Category columns"],
[
"The categories in the training data are the important categories. New categories follow a rule"
],
["One 0/1 column for each category value"],
_B,
_NOW,
"Region north/south becomes columns north = 1/0 and south = 1/0.",
related=["core.encode_levels"],
fits=_PREP,
),
"core.encode_levels": _a(
"Number each category",
"Gives each category value a number. Many models use this number to find a separate effect for each group.",
[
"A model needs group numbers (one effect for each store or market)"
],
["The model reads category labels itself"],
["Category columns or dimensions"],
[
"The step gives the numbers on the training data. New values follow a rule"
],
["An integer code for each category value"],
_I,
_NOW,
"Stores A, B, C become 0, 1, 2.",
related=["core.one_hot"],
fits=_PREP,
),
"core.missing_indicator": _a(
"Flag missing values",
"Adds a yes/no column that marks where a value was missing. Then a model can tell real values from filled values.",
[
"The fact that a value is missing gives you information"
],
["There are no gaps"],
["Columns that can have gaps"],
["Use this step before you fill the gaps"],
["One 0/1 column for each input column"],
_B,
_NOW,
"A missing price gets a flag of 1 next to the filled price.",
related=["core.impute"],
fits=_PREP,
),
"dl.embed": _a(
"Image features from a pretrained network",
"Changes each image into a short list of numbers with a pretrained network. A simple classifier after this step can then learn the classes from a few examples.",
[
"You have a pretrained network as a weights dataset",
"You want a fast or a Bayesian classifier on images with few labels",
],
[
"You have no pretrained weights that suit your images",
"The dataset is a large image collection (use the fine-tune classifier, or set a sample in data.collection)",
],
[
"An image dataset: pixels and, for the classifier, a label for each image",
"A weights dataset from a safetensors upload, given as a side table",
],
[
"The network learned features that also separate your classes",
"The network does not change: the step does not train it",
],
[
"A variable 'embedding' with one row of numbers for each image"
],
_I,
_SEC,
"Change 500 product photos into 64 numbers each, then classify them with logistic regression.",
related=[
"dl.finetune_classifier",
"sklearn.multiclass_classifier",
"tabular.image_logistic",
],
expert="A frozen mapi-jax ResNet backbone (GroupNorm, global average pooling) loaded from a weights side table; writes embedding(image, feature) float32 and records the weights sha256 in its state. Batches through the JIT-compiled feature function; runs on the CPU in every image that has mapi-jax.",
fits={
"goal": ["classify"],
"data_shape": ["images"],
},
),
}
Cards of the built-in preprocessing steps (mapi-steps) and of plugin steps
(dl.embed of mapi-jax).
SPLIT_ABOUTS: Final[dict[str, About]] = {
"core.time_holdout": _a(
"Test on the latest periods",
"Keeps back the most recent periods and fits the model on the rest. Then it measures how well the model predicts the periods that it did not see.",
[
"You forecast, or you want an honest error for recent periods"
],
["Your data does not have a time order"],
["An ordered dimension such as date"],
["The recent past is similar to the near future"],
[
"A training part and a test part, and holdout error metrics"
],
_B,
_NOW,
"With 104 weeks, the model learns on the first 96 weeks and the test uses the last 8 weeks.",
related=["core.as_of", "split.rolling_origin"],
fits={"goal": ["forecast", "validate_model"]},
),
"core.as_of": _a(
"Fit as of a past date",
"Fits the model as if today is a past date. Then it measures the predictions for the periods after that date.",
[
"You want to do a past decision point again",
"Data comes with a delay (use a gap)",
],
["Your data does not have a time order"],
["An ordered dimension and a cut-off date"],
[
"The cut-off leaves sufficient history for the fit"
],
[
"A training part up to the date and a test part after it"
],
_B,
_NOW,
"As of 2024-06-30 with a 2-week gap, the test uses July from week 3.",
related=[
"core.time_holdout",
"split.expanding_window",
],
fits={"goal": ["forecast", "validate_model"]},
),
"core.leave_group_out": _a(
"Test on unseen markets",
"Removes full markets or stores from the fit and measures how well the model predicts them.",
[
"You will use the model for new markets or stores"
],
["You only predict for markets that the model saw"],
["A group dimension with several levels"],
[
"The removed groups are similar to the other groups"
],
["A training part and the held-out groups"],
_I,
_NOW,
"Fit on 8 markets and test on the other 2.",
related=["split.geo_holdout"],
fits={
"goal": ["predict_values", "validate_model"],
"data_shape": ["panel"],
},
),
"core.last_per_group": _a(
"Test each group on its latest rows",
"Keeps back the last observations of each customer or store and tests the model on them.",
[
"Groups have different lengths of history, such as shopping trips for each household"
],
["Your data is one series"],
["A group dimension and an order in each group"],
[
"Each group has sufficient earlier rows to learn from"
],
[
"Training rows and the held-out last rows of each group"
],
_I,
_NOW,
"The test uses the last trip of each household that has a minimum of 3 trips.",
related=["core.time_holdout"],
fits={
"goal": [
"understand_choices",
"validate_model",
],
"data_shape": ["choices"],
},
),
"core.mask_block": _a(
"Hide random patches",
"Hides random blocks of the data from the fit and measures how well the model fills them in.",
[
"You want to test how the model fills gaps, not how it forecasts"
],
[
"You need a forecast error (hold out the latest periods instead)"
],
["One or more dimensions to cut blocks from"],
["The model accepts missing target values"],
[
"The data with hidden cells, and the positions of these cells"
],
_X,
_NOW,
"Hide 10% of the market-week cells in blocks of 4 weeks.",
related=["core.time_holdout"],
fits={"goal": ["validate_model"]},
),
"core.column": _a(
"Split by a column",
"Uses a column of the data, such as split = test, to select the holdout rows.",
["You already have a train and test assignment"],
[
"You forecast a time series (hold out the latest periods instead)"
],
[
"A column along the rows with the assignment, such as train and test"
],
[
"The holdout rows are similar to the rows that you will predict"
],
[
"A training part and the holdout rows, and holdout error metrics"
],
_B,
_NOW,
"An image upload has a split column with train and test. The test uses the 'test' images.",
related=["core.stratified", "core.leave_group_out"],
fits={
"goal": [
"classify",
"predict_values",
"validate_model",
]
},
),
"core.stratified": _a(
"Stratified random holdout",
"Holds out the same share of rows from each class, selected at random.",
["Classes have very different sizes"],
[
"You forecast a time series (hold out the latest periods instead)"
],
["A class column along the rows"],
["The rows are independent of each other"],
[
"A training part and a holdout with the same class shares, and holdout metrics"
],
_B,
_NOW,
"Hold out 25% of the images of each class. A class with 40 images gives 10 to the test.",
related=["core.column", "core.leave_group_out"],
fits={
"goal": [
"classify",
"predict_values",
"validate_model",
]
},
),
}
Cards of the built-in train/test splits (mapi_steps.splits).
MODEL_ABOUTS: Final[dict[str, About]] = {
"tabular.regressor": _a(
"Demand by store and week",
"Predicts a number for each store, product and week from prices, promotions and the calendar. It gives a low and a high estimate.",
[
"You have many stores or products and you want a forecast for each",
"Prices, promotions or stock explain demand",
],
[
"You need to add predictions into totals with a range (you cannot add ranges)",
"You have one short series",
],
[
"A table of the number to predict and its drivers, for each store and week"
],
[
"You know the future values of the drivers when you forecast",
"Stores that the model did not see behave like the average store",
],
[
"A median forecast with low and high values for each cell",
"Feature importance",
],
_B,
_SEC,
"Predict the units of next week for 300 stores from planned prices and promotions.",
related=["core.calendar", "core.one_hot"],
expert="Gradient-boosted quantile regression (xgboost or sklearn) on stacked rows. Coordinates become categorical features. Direct multi-horizon forecasting. Quantile representation (no cross-cell aggregation).",
fits={
"goal": ["predict_values", "forecast"],
"data_shape": ["panel", "table"],
"drivers": ["yes"],
"frequency": [
"daily",
"weekly",
"monthly",
"not_time",
],
},
),
"tabular.hierarchical_glm": _a(
"Group effects with sharing",
"Estimates a separate baseline for each group, such as a county or a store. Small groups get help from the data of the other groups.",
[
"Groups have very different quantities of data",
"You want to predict for groups that the model did not see",
],
["You have one series over time"],
[
"One row for each observation with a group label and predictors"
],
[
"Groups are sufficiently similar to share information",
"Effects are additive on the scale of the model",
],
[
"Effects for each group with ranges",
"Predictions for new groups",
],
_I,
_MIN,
"Radon levels for each house. If a county has few houses, its baseline moves towards the state average.",
related=["core.encode_levels"],
expert="Bayesian hierarchical GLM (PyMC, NUTS). Partially pooled group intercepts and optional slopes. Group-level covariates in the mean. The model draws new levels from the fitted hyperprior. Families: normal, Student-t, Poisson, Bernoulli, negative binomial, zero-inflated Poisson and negative binomial (counts with many zeros), beta (shares between 0 and 1), gamma (positive amounts) and ordinal (ordered categories, cumulative logit).",
fits={
"goal": ["explain_drivers", "predict_values"],
"data_shape": ["table", "panel"],
"uncertainty": ["essential", "nice"],
"frequency": ["not_time"],
},
),
"choice.multinomial_logit": _a(
"Customer choice",
"Explains which product a customer selects on each shopping trip, and how price and promotions change that choice.",
[
"You have purchase records with the price of each product on offer",
"You want to test a price change before you make it",
],
[
"You only have total sales, not individual choices"
],
[
"One row for each trip: the product that the customer selected, and the price and promotion of each product"
],
[
"Customers select the option that they like best for the given price and promotion",
"Differences in taste between households are stable over time",
],
[
"How sensitive the choices are to price and promotion",
"Predicted choice shares",
"What-if shares for new prices",
],
_I,
_MIN,
"A 10% decrease in the price of Brand A increases its share from 30% to 36%.",
related=["core.last_per_group"],
expert="Conditional (McFadden) logit over alternative-specific covariates with household random effects. Bayesian-bootstrap draws over households.",
refs=[
(
"McFadden (1974), Conditional logit analysis of qualitative choice behavior",
None,
)
],
fits={
"goal": ["understand_choices"],
"data_shape": ["choices"],
"uncertainty": ["essential", "nice"],
},
),
"ts.gaussian_hmm": _a(
"Hidden states over time",
"Finds a small number of hidden states, such as calm periods and busy periods, that a series changes between. It tells you the probable state of each period.",
[
"Your series seems to change between distinct modes"
],
[
"Behaviour changes smoothly, not in jumps",
"You have very short sequences",
],
["One or more sequences of measurements over time"],
[
"Each state has its own typical level and spread",
"The probability of a change of state depends only on the current state",
],
[
"The probable state of each period",
"How frequently the states change",
"Forecasts",
],
_X,
_MIN,
"The track of an animal divides into rest phases and travel phases.",
related=["ts.markov_switching"],
expert="K-state Gaussian HMM. The forward algorithm in PyMC marginalises the states. The terminal state is stored for forecast and filter. Ordering fixes label switching.",
fits={
"goal": ["find_regimes"],
"data_shape": ["one_series", "panel"],
"uncertainty": ["essential", "nice"],
},
),
"ts.hsgp_trend": _a(
"Smooth trend forecast",
"Forecasts one regular series with a smooth, flexible trend and a repeating seasonal pattern. It gives a range of probable values.",
[
"You forecast one weekly or daily series with a trend that bends over time",
"You want an honest range around the forecast",
],
[
"You have many related series (forecast them together or for each store)",
"You need a forecast further ahead than the model permits without a new fit",
],
["One number over time at a regular frequency"],
[
"The trend changes gradually",
"The seasonal pattern repeats",
],
[
"A forecast with a range",
"Trend and seasonal parts",
],
_B,
_MIN,
"Forecast weekly website visits 26 weeks ahead with a 90% range.",
related=["core.fourier", "ts.ets", "ts.sarimax"],
expert="Hilbert-space Gaussian process (HSGP) trend with optional linear drift and Fourier seasonality (PyMC, NUTS). Forecasts are limited to max_extension periods.",
refs=[
(
"Riutort-Mayol et al. (2022), Practical Hilbert space approximate Bayesian Gaussian processes",
"https://arxiv.org/abs/2004.11408",
)
],
fits={
"goal": ["forecast"],
"data_shape": ["one_series"],
"frequency": ["daily", "weekly", "monthly"],
"history": ["medium", "long"],
"uncertainty": ["essential", "nice"],
},
),
"mmm.bayesian": _a(
"Marketing mix model",
"Measures how much each marketing channel adds to your sales, in total or for each market. It also shows what occurs if you move budget between channels.",
[
"You have two or more years of weekly sales and spend for each channel",
"You want to know the return of each channel and plan budgets",
"You have results for each market or product and you want one model for all of them",
"Your result is a count (orders, sign-ups) or a share between 0 and 1",
"The effect of a channel changes over time, for example when an advertisement gets old",
],
[
"Spend changes very little over time (the model cannot measure the effect)",
"You have less than a year of history",
"You need answers in seconds (use the fast frequentist version)",
],
[
"Weekly (or daily) sales, spend for each channel, and optional controls such as price"
],
[
"Advertising has an effect that continues for some time, and diminishing returns",
"The controls include the factors that change both spend and sales",
],
[
"Contribution and return for each channel, with ranges",
"Response curves",
"Budget plans and what-if answers",
"The effect of each channel over time, when the effect can change",
],
_I,
_LONG,
"TV returns 2.1 euros for each euro of spend (range 1.4 to 2.9) and search returns 3.4.",
related=[
"mmm.frequentist",
"mmm.nested",
"mmm.multivariate",
"core.resample",
],
expert="mmm-framework BayesianMMM: geometric/Weibull adstock, Hill or logistic saturation, trend options, Fourier seasonality, and hierarchical pooling over geos and products (MFF panels). Priors come from business beliefs. Lift tests are a calibration likelihood. Time-varying channel coefficients are a random walk on the log coefficient, with a forecast policy (random walk, hold last, mean) after the fit window. Poisson and negative binomial (log link) and beta (logit link) likelihoods split the expected KPI into channel parts in proportion to their link-scale terms; ROI priors and lift tests apply only to the normal and Student-t likelihoods. NUTS through PyMC, nutpie or numpyro.",
refs=[
(
"mmm-framework documentation",
"https://redam94.github.io/mmm-framework/",
),
(
"Jin et al. (2017), Bayesian methods for media mix modeling with carryover and shape effects",
None,
),
],
fits={
"goal": ["explain_marketing", "plan_budget"],
"data_shape": ["one_series", "panel"],
"frequency": ["daily", "weekly"],
"history": ["medium", "long"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"mmm.nested": _a(
"Marketing mix with steps in between",
"Measures marketing effects that go through an intermediate result. For example, ads bring visits to your website, and the visits bring sales.",
[
"You have an intermediate measure, such as visits, searches or awareness",
"You want to know how much of the effect of a channel goes through it",
"You measure the intermediate result only in some periods, for example with a monthly survey",
],
[
"You only have sales and spend (use the standard marketing mix model)",
"You have results for each market or product (this model uses one national series)",
"Price, promotions or other drivers change sales much (this model has no control variables)",
],
[
"One national series of sales and spend for each channel over time",
"One column for each intermediate measure. A measure can have empty periods",
],
[
"Advertising has an effect that continues for some time, and diminishing returns",
"The intermediate measure carries all or part of the effect of a channel",
"No other driver changes both the spend and the sales",
],
[
"Direct and indirect contribution for each channel, with ranges",
"The part of the effect of each channel that goes through each measure",
"Fitted values of the intermediate measures",
"Response curves, budget plans and what-if answers",
],
_X,
_LONG,
"Search ads increase visits, and the visits bring 40% of the effect of search on online sales.",
related=["mmm.bayesian", "mmm.multivariate"],
expert="mmm-framework mmm_extensions NestedMMM: normalised geometric adstock (8 lags) and exponential saturation shared by the mediator and outcome equations. A measured mediator has Gaussian noise on its non-missing periods; a latent one is not observed (only the product of its two paths is identified). Outcome likelihood normal or Student-t, standardised. National data only, no control variables. Lift tests calibrate the total effect of a channel. NUTS through PyMC or nutpie, or ADVI.",
refs=[
(
"mmm-framework documentation",
"https://redam94.github.io/mmm-framework/",
)
],
fits={
"goal": ["explain_marketing", "plan_budget"],
"data_shape": ["several_series"],
"frequency": ["daily", "weekly"],
"history": ["medium", "long"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"mmm.multivariate": _a(
"Marketing mix for several results",
"Measures marketing effects on two or more results at the same time, such as the sales of related products. It also shows how the sales of one product move with the sales of a different product.",
[
"You have two or more results over time, such as the sales of related products",
"One marketing budget acts on all of the results",
"You think that the sales of one product go up or down with the sales of a different product",
],
[
"Only one result is important to you (use the standard marketing mix model)",
"You have results for each market (this model uses national data)",
"You must prove that one product takes sales from a different product (the model shows associations, not causes)",
],
[
"One national column for each result and spend for each channel over time",
"The results use the same units, or you give a value for one unit of each result",
],
[
"Advertising has an effect that continues for some time, and diminishing returns",
"The effects between the results are steady over the period",
"No other driver changes both the spend and the results",
],
[
"Contribution of each channel to each result, with ranges",
"Effects between the results (halo or cannibalization)",
"Response curves and budget plans for a weighted total of the results",
],
_X,
_LONG,
"The advertising of the premium range also increases the sales of the basic range by 5%.",
related=["mmm.bayesian", "mmm.nested"],
expert="mmm-framework mmm_extensions MultivariateMMM: one equation per outcome with shared or separate adstock, exponential saturation, trend and Fourier seasonality; a multivariate normal observation with an LKJ correlation prior. Cross-effects (psi) act on the observed source outcome, so they are associations that are confounded with the residual correlation. Scores, response curves and budget plans use the reduced form (I - P)^-1, so a channel that moves one outcome also moves the others through psi. National data only, no control variables, no lift-test calibration and no promotion modulation.",
refs=[
(
"mmm-framework documentation",
"https://redam94.github.io/mmm-framework/",
)
],
fits={
"goal": ["explain_marketing", "plan_budget"],
"data_shape": ["several_series"],
"frequency": ["daily", "weekly"],
"history": ["medium", "long"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"mmm.frequentist": _a(
"Fast marketing mix model",
"Gives a quick first estimate of the contribution of each channel in seconds. Its ranges (from resampled data) are simpler than the uncertainty of a full Bayesian model.",
[
"You need a fast first look before a full model",
"You want to compare many set-ups quickly",
],
[
"Decisions depend on the full uncertainty of the answer (use the Bayesian model)",
"You want to add business beliefs or lift tests",
],
["Weekly sales and spend for each channel"],
[
"The model searches a grid for the carry-over and saturation settings",
"Channel effects are not negative",
],
[
"Contribution and return for each channel, with bootstrap ranges"
],
_B,
_SEC,
"A first result in 5 seconds: TV 28% of sales, search 12%.",
related=["mmm.bayesian"],
expert="mmm-framework frequentist ridge / constrained least squares with a grid search over transform parameters. Block-bootstrap replicates (a minimum of 20) are the draws of each output, so its ranges are confidence intervals, not posteriors.",
refs=[
(
"mmm-framework documentation",
"https://redam94.github.io/mmm-framework/",
)
],
fits={
"goal": ["explain_marketing"],
"data_shape": ["one_series", "panel"],
"uncertainty": ["no", "nice"],
"drivers": ["yes"],
},
),
"ts.sarimax": _a(
"Forecast with past values and drivers",
"Forecasts a series from its recent past, its seasonal pattern and outside drivers that you know in advance, such as planned prices or holidays.",
[
"You forecast one regular series some periods ahead",
"Known future drivers (prices, promotions, holidays) are important",
],
[
"You have less than about 50 periods",
"Two or more series change each other (use VAR)",
],
[
"One number over time at a regular frequency, and optional driver columns that you know for the future"
],
[
"After differencing, the series varies around a stable level",
"Relationships stay the same over time",
],
[
"A forecast with a range",
"Effects of the drivers",
"Residual checks",
],
_I,
_SEC,
"Forecast weekly orders 12 weeks ahead from the orders of recent weeks and planned discounts.",
related=[
"ts.arima",
"search.arma_order",
"test.adf",
"transform.difference",
],
expert="statsmodels SARIMAX (state space): ARIMA(p,d,q)(P,D,Q)s with exogenous regressors. Exact likelihood through the Kalman filter. Prediction intervals as quantiles. Forecasts need future exog.",
refs=[
_TS_REF,
(
"Hyndman & Athanasopoulos, Forecasting: Principles and Practice (ch. 9-10)",
"https://otexts.com/fpp3/",
),
],
fits={
"goal": ["forecast", "explain_drivers"],
"data_shape": ["one_series"],
"frequency": ["daily", "weekly", "monthly"],
"history": ["medium", "long"],
"drivers": ["yes", "no"],
"uncertainty": ["essential", "nice", "no"],
},
),
"ts.arima": _a(
"Forecast from past values",
"Forecasts a series from its recent past and trend, without outside drivers.",
[
"You have only the series",
"You want a reliable, well-known forecast method",
],
[
"Outside drivers explain most of the movement (use the version with drivers)"
],
["One number over time at a regular frequency"],
[
"After differencing, the series varies around a stable level"
],
["A forecast with a range", "Residual checks"],
_I,
_SEC,
"Forecast monthly sales 6 months ahead from the sales of recent years.",
related=[
"ts.sarimax",
"search.arma_order",
"test.adf",
],
expert="statsmodels ARIMA(p,d,q)(P,D,Q)s (state space) without exog.",
refs=[_TS_REF],
fits={
"goal": ["forecast"],
"data_shape": ["one_series"],
"drivers": ["no"],
"frequency": ["daily", "weekly", "monthly"],
"history": ["medium", "long"],
},
),
"ts.var": _a(
"Series that drive each other",
"Models two or more series that change each other over time, such as the prices and sales of two brands. It forecasts them together.",
[
"Two or more series move together and change each other",
"You want to see how a shock to one series goes to the other series",
],
[
"You have one series",
"The series have the same trend permanently (use the error-correction model)",
],
[
"Two or more numbers over time at the same frequency"
],
[
"Each series varies around a stable level, after differencing if necessary",
"Effects work through past values",
],
[
"Joint forecasts with ranges",
"How a shock goes through the series (impulse responses)",
"Which series help to predict which series (Granger tests)",
],
_X,
_SEC,
"A price decrease by brand A decreases the sales of brand B for three weeks.",
related=[
"ts.varmax",
"ts.vecm",
"test.granger",
"search.var_lag_order",
],
expert="statsmodels VAR(p): OLS for each equation. Lag order by AIC/BIC/HQIC. Impulse responses and forecast-error variance decomposition. Granger causality tests.",
refs=[
_TS_REF,
(
"Lutkepohl, New Introduction to Multiple Time Series Analysis",
None,
),
],
fits={
"goal": ["compare_series", "forecast"],
"data_shape": ["several_series"],
"history": ["medium", "long"],
},
),
"ts.varmax": _a(
"Series that drive each other, with drivers",
"Models two or more series that change each other and also respond to outside drivers that you know in advance. It forecasts them together.",
[
"Two or more series change each other, and outside drivers are also important"
],
[
"You have one series",
"You have many series and little history",
],
[
"Two or more numbers over time, and driver columns that you know for the future"
],
[
"Series vary around stable levels",
"Driver effects are steady",
],
[
"Joint forecasts with ranges",
"Driver effects",
"Impulse responses",
],
_X,
_MIN,
"Forecast the sales of two brands from the history of each brand and planned promotions.",
related=["ts.var", "ts.sarimax"],
expert="statsmodels VARMAX(p,q) with exogenous regressors (VARX when q = 0), estimated by maximum likelihood in state-space form.",
refs=[_TS_REF],
fits={
"goal": [
"compare_series",
"forecast",
"explain_drivers",
],
"data_shape": ["several_series"],
"drivers": ["yes"],
"history": ["medium", "long"],
},
),
"ts.vecm": _a(
"Series tied together in the long run",
"Models series that wander but stay tied together in the long run, such as the prices of close substitutes. It shows how fast they go back after a gap opens.",
[
"The series have a trend but keep a stable long-run relationship"
],
[
"The series do not have a trend",
"A long-run tie is not probable",
],
["Two or more numbers with a trend over time"],
[
"A stable long-run relationship exists (the Johansen test does a check of this)"
],
[
"Long-run relationships",
"Speed of adjustment",
"Joint forecasts with ranges",
],
_X,
_SEC,
"When the price of a competitor goes 5% above your price, half of the gap closes in 4 weeks.",
related=[
"test.johansen",
"search.coint_rank",
"ts.var",
"ts.ecm",
],
expert="statsmodels VECM with cointegration rank from the Johansen trace/max-eigenvalue tests. You can select the deterministic terms.",
refs=[
_TS_REF,
(
"Johansen (1991), Estimation and hypothesis testing of cointegration vectors",
None,
),
],
fits={
"goal": ["compare_series", "forecast"],
"data_shape": ["several_series"],
"history": ["long", "medium"],
},
),
"ts.ecm": _a(
"Long-run and short-run effects",
"Separates the short-term effect of a driver from its long-term effect on one series. It also tests if a long-run relationship exists.",
[
"One result and some drivers have the same trend",
"You want both the immediate effect and the lasting effect of a driver",
],
[
"The result and the drivers do not have a trend (a regression with lags is sufficient)"
],
["One result and one or more drivers over time"],
[
"The drivers change the result, but the result does not change the drivers"
],
[
"Short-run and long-run effects",
"Speed of adjustment",
"A bounds test result",
],
_X,
_SEC,
"A 1% price increase decreases sales by 0.4% immediately and by 1.1% in the long run.",
related=[
"ts.ardl",
"ts.vecm",
"test.engle_granger",
],
expert="Single-equation error-correction model in UECM form (statsmodels UECM). Pesaran-Shin-Smith bounds test for a levels relationship.",
refs=[
_TS_REF,
(
"Pesaran, Shin & Smith (2001), Bounds testing approaches",
None,
),
],
fits={
"goal": ["explain_drivers", "forecast"],
"data_shape": ["one_series"],
"drivers": ["yes"],
"history": ["medium", "long"],
},
),
"ts.ardl": _a(
"Effects that build up over time",
"Explains a series from its past and from the current and past values of drivers. You can see how an effect increases over several periods.",
[
"The effect of a driver spreads over several periods"
],
[
"Effects are only immediate (a simple regression is sufficient)"
],
["One result and one or more drivers over time"],
[
"The drivers change the result, but the result does not change the drivers",
"The series vary around stable levels or share a long-run relationship",
],
[
"Effects for each lag",
"Long-run multipliers",
"Forecasts for given future drivers",
],
_I,
_SEC,
"A promotion increases sales by 8% in its week and by 3% in the next week.",
related=["ts.ecm", "ts.sarimax"],
expert="statsmodels ARDL with lag orders that an information criterion selects (ardl_select_order).",
refs=[_TS_REF],
fits={
"goal": ["explain_drivers", "forecast"],
"data_shape": ["one_series"],
"drivers": ["yes"],
},
),
"ts.ets": _a(
"Exponential smoothing forecast",
"Forecasts a series and gives recent values more weight than older values. It has an optional trend and seasonality, and it is quick and robust.",
[
"You need a fast, reliable baseline forecast",
"The level, trend and seasonality change slowly",
],
["Outside drivers explain most of the movement"],
["One number over time at a regular frequency"],
[
"Recent history is the best guide to the near future"
],
[
"A forecast with a range",
"Level, trend and seasonal parts",
],
_B,
_SEC,
"Forecast monthly demand 12 months ahead with a damped trend.",
related=[
"ts.theta",
"ts.arima",
"evaluate.baseline",
],
expert="statsmodels ETSModel (error, trend, seasonal; additive or multiplicative; damped trend), with prediction intervals from simulation.",
refs=[
_TS_REF,
(
"Hyndman et al., Forecasting with Exponential Smoothing",
None,
),
],
fits={
"goal": ["forecast"],
"data_shape": ["one_series"],
"drivers": ["no"],
"frequency": ["daily", "weekly", "monthly"],
},
),
"ts.unobserved_components": _a(
"Trend, season and noise",
"Divides a series into a level and a trend that change slowly, a seasonal pattern and noise. It forecasts the series from these parts.",
[
"You want to see the underlying trend without the noise",
"The trend and the season change slowly over time",
],
[
"You only need a quick baseline (exponential smoothing is simpler)"
],
["One number over time, and optional drivers"],
["Each part changes smoothly and independently"],
[
"Smoothed level, trend and season",
"Forecasts with ranges",
],
_I,
_SEC,
"Weekly sales divide into an increasing trend, a Christmas peak and noise from week to week.",
related=["ts.ets", "ts.hsgp_trend"],
expert="statsmodels UnobservedComponents (local level/trend, stochastic seasonal, cycle, exog), estimated by maximum likelihood with the Kalman smoother.",
refs=[
_TS_REF,
(
"Harvey, Forecasting, Structural Time Series Models and the Kalman Filter",
None,
),
],
fits={
"goal": ["forecast", "explain_drivers"],
"data_shape": ["one_series"],
},
),
"ts.dynamic_factor": _a(
"Common movements across many series",
"Finds a few hidden forces that move many series together, such as the general economy behind the sales in many categories.",
[
"You have many related series and you want their shared drivers"
],
["You have one or two series"],
["Many numbers over time at the same frequency"],
[
"Some common factors explain most of the joint movement"
],
[
"The common factors over time",
"How much each series follows them",
"Forecasts",
],
_X,
_MIN,
"Twenty series of category sales follow one 'consumer mood' factor.",
related=["ts.var"],
expert="statsmodels DynamicFactor (state space; the factors follow a VAR).",
refs=[_TS_REF],
fits={
"goal": ["compare_series", "forecast"],
"data_shape": ["several_series", "panel"],
},
),
"ts.markov_switching": _a(
"Changes between regimes",
"Finds if a series changes between regimes, such as periods of high growth and low growth. It estimates the behaviour in each regime.",
[
"Behaviour seems to jump between distinct modes",
"You want the probability of each mode now",
],
["Changes are gradual", "You have little history"],
["One number over time, and optional drivers"],
[
"There is a small number of regimes. A change of regime depends only on the current regime"
],
[
"The probability of each regime for each period",
"Behaviour in each regime",
],
_X,
_SEC,
"Demand changes between a normal regime and a regime that promotions cause.",
related=["ts.gaussian_hmm", "test.cusum"],
expert="statsmodels MarkovRegression / MarkovAutoregression (Hamilton filter, EM then BFGS).",
refs=[
_TS_REF,
(
"Hamilton (1989), A new approach to the economic analysis of nonstationary time series",
None,
),
],
fits={
"goal": ["find_regimes"],
"data_shape": ["one_series"],
},
),
"ts.theta": _a(
"Simple strong baseline",
"Forecasts a series with a combination of a straight-line trend and smoothed recent values. This simple method is hard to beat.",
[
"You want a strong, simple baseline to compare with other models"
],
[
"Outside drivers are very important",
"You need detailed explanations",
],
[
"One number over time (the model removes the seasonality and puts it back automatically)"
],
["The recent trend continues"],
["A forecast with a range"],
_B,
_NOW,
"A theta forecast of monthly sales that other models must beat before you use them.",
related=["ts.ets", "evaluate.baseline"],
expert="statsmodels ThetaModel (Assimakopoulos-Nikolopoulos; equivalent to SES with drift) with optional seasonal decomposition.",
refs=[_TS_REF],
fits={
"goal": ["forecast"],
"data_shape": ["one_series"],
"drivers": ["no"],
},
),
"ts.bsts": _a(
"Bayesian trend, season and drivers",
"Divides one series into a level, a trend, seasonal patterns and the effect of drivers that you know. It forecasts the series with a range of probable values.",
[
"You forecast one regular series and you want an honest range",
"The level, the trend or the seasonal pattern can change over time",
"Some values are missing (the model continues over the gaps)",
"You want to update the forecast with new periods without a new fit",
],
[
"You have many related series (this model takes one series)",
"The effect of a driver changes over time (the effect is the same in all periods)",
"You need results in seconds",
],
[
"One number over time at a regular frequency",
"Optional drivers (for example the price). You must know their future values to forecast.",
],
[
"The level and the trend change by small random steps",
"Each driver has the same effect in all periods",
],
[
"A forecast with a range",
"The level and the seasonal parts over time",
"The effect of each driver with a range",
"A filter mode that adds new periods to the fit state",
],
_I,
_MIN,
"Monthly sales divide into a slow increase, a December peak and a price effect of -0.3 units for each euro.",
related=[
"ts.unobserved_components",
"ts.hsgp_trend",
"causal.impact",
],
expert="Bayesian structural time series on pymc-extras statespace: local level, local linear or smooth trend, dummy or Fourier seasonal components (fixed or with innovations), static regression (normal or horseshoe prior; the regression states start with zero variance, so the coefficients are static). Exact Kalman likelihood (NUTS). Scores re-run a numpy Kalman filter for each kept draw: one-step-ahead inside the fit, a forecast from the terminal state after it, and a 'filter' mode that continues the fit with new data. Leave-one-out is not defined.",
refs=[
(
"Harvey (1989), Forecasting, Structural Time Series Models and the Kalman Filter",
None,
),
(
"Durbin and Koopman (2012), Time Series Analysis by State Space Methods",
None,
),
(
"Scott and Varian (2014), Predicting the present with Bayesian structural time series",
None,
),
],
fits={
"goal": ["forecast"],
"data_shape": ["one_series"],
"frequency": ["daily", "weekly", "monthly"],
"history": ["medium", "long"],
"uncertainty": ["essential", "nice"],
},
),
"causal.impact": _a(
"Effect of a campaign on one series",
"Measures the effect of a campaign, a launch or a change on one series. It compares the actual values with the values that the model predicts without the change.",
[
"A campaign, a launch or a price change started on a known date",
"You have other series that the change did not touch (for example the sales in other countries, or searches for the category)",
"You want the effect with a range, for each period and in total",
],
[
"The change also changed the control series",
"Other events occurred at the same time as the change",
"You have fewer than about 10 periods before the change",
"You have the same series for many markets (use the synthetic control)",
],
[
"One number over time at a regular frequency, before and after the change",
"The start date of the change, and an end date if it stopped",
"Optional control series for the full period",
],
[
"Without the change, the series continues as the model of the period before it predicts",
"The change did not touch the control series",
],
[
"Actual values and the values without the change, with a range",
"The effect in each period and the total effect",
"The probability that the effect is more than zero",
"A chart of the fit before the change",
],
_I,
_MIN,
"A TV campaign from 1 March added 4,200 sales in 8 weeks (range 2,900 to 5,500).",
related=[
"ts.bsts",
"causal.synthetic_control",
"causal.did",
],
expert="CausalImpact (Brodersen et al. 2015) on the ts.bsts core: a Bayesian structural time series with the controls as static regressors is fitted on the pre-period only. The counterfactual is the one-step-ahead prediction in the pre-period and a predictive forecast from the terminal state after the start (common random numbers). The effect is actual minus counterfactual, per draw; the cumulative effect sums the post cells. Interrupted time series is the case without controls.",
refs=[
(
"Brodersen et al. (2015), Inferring causal impact using Bayesian structural time-series models, Annals of Applied Statistics",
"https://doi.org/10.1214/14-AOAS788",
)
],
fits={
"goal": ["measure_impact"],
"data_shape": ["one_series", "several_series"],
"frequency": ["daily", "weekly", "monthly"],
"uncertainty": ["essential", "nice"],
},
),
"causal.synthetic_control": _a(
"Geo-lift test",
"Measures the effect of a campaign in some markets. It compares each test market with a weighted mix of the other markets that did not get the campaign.",
[
"You did a test in some markets and kept the other markets as they were",
"You have the same number for many markets, before and during the test",
"You want a lift test to calibrate a marketing mix model",
],
[
"The test markets are very different from all other markets (for example the largest market)",
"The campaign also changed the other markets",
"You have fewer than about 10 periods before the test",
],
[
"The same number for each market over time, before and during the test",
"The names of the test markets, and the start and end dates of the test",
"At least two markets without the campaign, with no missing values before the test",
],
[
"Before the test, a weighted mix of the other markets follows each test market",
"Without the campaign, this mix continues to follow the test market",
],
[
"Actual values and the values without the campaign for each test market, with a range",
"The total effect and the effect in each period",
"The weight of each market in the mix",
"A lift test for the marketing mix model",
],
_I,
_MIN,
"TV in Leeds and Bristol for 6 weeks added 3% to sales (range 1% to 5%), compared with a mix of 10 other markets.",
related=[
"causal.impact",
"causal.did",
"mmm.bayesian",
],
expert="Bayesian synthetic control (Abadie, Diamond and Hainmueller 2010): for each treated unit, y_pre ~ Normal|StudentT(alpha + Y_donor,pre @ w, sigma) on the pre-period, with w on the simplex (Dirichlet) or a regularised horseshoe (can extrapolate). Units are divided by their pre-period mean. The counterfactual is the predictive draw alpha + Y_donor @ w + noise. There are no placebo tests and no test design (market choice, power) yet. A lift test from several markets is national.",
refs=[
(
"Abadie, Diamond and Hainmueller (2010), Synthetic control methods for comparative case studies, JASA",
"https://doi.org/10.1198/jasa.2009.ap08746",
),
(
"Kim, Lee and Gupta (2020), Bayesian synthetic control methods, Journal of Marketing Research",
"https://doi.org/10.1177/0022243720936230",
),
],
fits={
"goal": ["measure_impact"],
"data_shape": ["panel"],
"frequency": ["daily", "weekly", "monthly"],
"uncertainty": ["essential", "nice"],
},
),
"causal.did": _a(
"Before and after, with a comparison group",
"Measures the effect of a change that some stores or markets got and others did not. It shows the effect for each period before and after the change.",
[
"Some units got a change and other units never got it",
"The units got the change at the same time or at different times",
"You want to see if the groups moved the same way before the change",
],
[
"All units got the change",
"The units got the change at different times and the effect is different for each group (the result can be wrong)",
"You want to forecast (the time effects are only for the periods of the data)",
],
[
"The same number for each unit over time",
"A column that is 1 from the period when a unit got the change, else 0",
"At least one unit that never got the change",
],
[
"Without the change, all units move in parallel",
"A unit does not stop the change after it starts",
],
[
"The average effect of the change with a range",
"The effect for each period before and after the change",
"A warning when the units got the change at different times",
],
_I,
_MIN,
"A new store layout in 12 of 30 stores added 5% to monthly sales, and the stores moved together before the change.",
related=[
"causal.synthetic_control",
"causal.impact",
],
expert="Bayesian two-way fixed effects: unit and time effects (ZeroSumNormal; time effects iid or a random walk) plus event-time dummies (reference period -1, endpoints binned at -leads/+lags) or one treatment dummy (design 'did'). Under staggered adoption with effects that differ between cohorts, TWFE estimates can be biased (Goodman-Bacon 2021, Sun and Abraham 2021): the model emits a STAGGERED_ADOPTION warning. Counterfactual mode sets treatment to 0.",
refs=[
(
"Goodman-Bacon (2021), Difference-in-differences with variation in treatment timing, Journal of Econometrics",
"https://doi.org/10.1016/j.jeconom.2021.03.014",
),
(
"Sun and Abraham (2021), Estimating dynamic treatment effects in event studies with heterogeneous treatment effects, Journal of Econometrics",
"https://doi.org/10.1016/j.jeconom.2020.09.006",
),
],
fits={
"goal": ["measure_impact"],
"data_shape": ["panel"],
"frequency": ["daily", "weekly", "monthly"],
"uncertainty": ["essential", "nice"],
},
),
"sklearn.tabular_regressor": _a(
"Predict a number from a table",
"Predicts a number for each row of a table, such as a store and week, from the other columns. It uses a well-known machine-learning method that you select.",
[
"You have many rows with drivers (price, promotion, weather) and you want one best estimate for each row",
"You want to see how much each driver adds to a prediction",
],
[
"You need a range of probable values (use the quantile, bootstrap or conformal versions)",
"You have one short series over time",
],
[
"A table of the number to predict and the columns that explain it"
],
[
"The rows that you predict are similar to the rows that the model learned from",
"You know the future values of the drivers when you forecast",
],
[
"One prediction for each row",
"The contribution of each driver (linear and tree models)",
],
_B,
_SEC,
"Predict the units of next week for each store and product from planned prices, with gradient boosting.",
related=[
"sklearn.tabular_quantiles",
"sklearn.tabular_conformal",
"core.one_hot",
],
expert="An allow-listed scikit-learn or xgboost regressor on rows stacked from the labelled dims (coordinates become features). Point representation. Exact linear or counterfactual-zeroing contributions. The server stores models as skops or xgboost JSON.",
fits={
"goal": ["predict_values", "explain_drivers"],
"data_shape": ["table", "panel"],
"drivers": ["yes"],
"uncertainty": ["no"],
},
),
"sklearn.tabular_quantiles": _a(
"Predict a number with a low and high estimate",
"Predicts a low, middle and high value for each row of a table. These values show how certain the prediction is.",
[
"You need a best case and a worst case for each store, product or customer",
"Stock or staff decisions depend on the high end or the low end",
],
[
"You need to add predictions into totals with a range (you cannot add the ranges of rows)"
],
[
"A table of the number to predict and the columns that explain it"
],
[
"The rows that you predict are similar to the rows that the model learned from"
],
["A low, middle and high prediction for each row"],
_I,
_SEC,
"Plan stock at the 90% level: the high estimate of the demand of next week for each store.",
related=[
"sklearn.tabular_regressor",
"sklearn.tabular_bootstrap",
],
expert="xgboost multi-quantile regression or one scikit-learn quantile model for each level. Quantile representation (you cannot aggregate the quantiles of cells across cells).",
fits={
"goal": ["predict_values", "forecast"],
"data_shape": ["table", "panel"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"sklearn.tabular_bootstrap": _a(
"Predict a number with ranges you can add up",
"Fits the same method many times on resampled data. This gives ranges that stay correct when you add predictions into totals.",
[
"You need a range for totals (all stores, a full region)",
"The data has groups (stores) that the resample must keep together",
],
[
"You need results in seconds on a very large table (it fits many models)"
],
[
"A table of the number to predict and the columns that explain it"
],
[
"The rows that you predict are similar to the rows that the model learned from",
"A resample of full groups shows how uncertain the model is",
],
[
"Many joint simulated predictions for each row",
"Ranges for all totals",
],
_I,
_MIN,
"A range for the total sales of next month across 300 stores.",
related=[
"sklearn.tabular_quantiles",
"sklearn.tabular_regressor",
],
expert="A bootstrap ensemble (with an optional resample of full groups with 'by'). Each replicate is a joint draw, so aggregation is valid (draws representation).",
fits={
"goal": ["predict_values", "forecast"],
"data_shape": ["table", "panel"],
"drivers": ["yes"],
"uncertainty": ["essential"],
},
),
"sklearn.tabular_conformal": _a(
"Predict a number with a guaranteed range",
"Predicts a number for each row with a range. The range contains the true value about as frequently as you ask, such as nine times in ten.",
[
"You need ranges with a coverage that you can trust for each row",
"You do not want to make assumptions about the shape of the errors",
],
[
"You need ranges for totals (use the bootstrap version)",
"You have very little data (the model keeps a part of it to set the size of the ranges)",
],
[
"A table of the number to predict and the columns that explain it"
],
[
"New rows behave like the rows that the model kept to set the size of the ranges"
],
["One prediction and a range for each row"],
_I,
_SEC,
"Ranges that contain the demand of next week for 90% of store-product pairs.",
related=[
"sklearn.tabular_quantiles",
"sklearn.tabular_regressor",
],
expert="Split-conformal prediction intervals at level 1 - alpha with a calibration fraction. Marginal coverage for each cell. Interval representation (you cannot aggregate it).",
fits={
"goal": ["predict_values"],
"data_shape": ["table", "panel"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"sklearn.tabular_classifier": _a(
"Predict yes or no",
"Estimates the probability that something occurs for each row of a table. For example, a customer buys again or an order arrives late.",
[
"The result is yes or no for each row",
"You want a probability to put rows in order or to select rows",
],
[
"The result is a number",
"The result has more than two classes (use the multiclass classifier)",
],
[
"A table with a yes/no column and the columns that explain it"
],
[
"The rows that you score are similar to the rows that the model learned from"
],
["The probability of 'yes' for each row"],
_B,
_SEC,
"The probability that each customer orders again next month, to decide who gets a voucher.",
related=[
"sklearn.multiclass_classifier",
"sklearn.tabular_regressor",
"choice.multinomial_logit",
],
expert="An allow-listed scikit-learn, xgboost, LightGBM or CatBoost binary classifier on stacked rows (LightGBM and CatBoost need the mapi-sklearn[gbm] extra, which the base-ml image has). Point representation of the positive-class probability.",
fits={
"goal": [
"predict_values",
"understand_choices",
"classify",
],
"data_shape": ["table", "panel", "choices"],
"drivers": ["yes"],
},
),
"sklearn.multiclass_classifier": _a(
"Put rows into classes",
"Estimates the probability of each class for each row of a table. For example, the product category of an order or the reason for a support call.",
[
"The result has three or more classes for each row",
"You want a probability for each class, not only the most probable class",
],
[
"The result is a number",
"The result is yes or no (use the yes-or-no classifier)",
"Some classes have only a few rows",
],
[
"A table with a class column (text or whole numbers) and the columns that explain it"
],
[
"The rows that you score are similar to the rows that the model learned from",
"The classes at scoring are the classes that the model learned",
],
[
"The probability of each class for each row",
"ROC and precision-recall curves for each class, and a confusion matrix",
],
_B,
_SEC,
"Put each support ticket into one of 6 topics, and send the tickets with a low probability to a person.",
related=[
"sklearn.tabular_classifier",
"tabular.image_logistic",
],
expert="An allow-listed multiclass classifier (logistic regression, random forest, histogram gradient boosting, xgboost, LightGBM or CatBoost; LightGBM and CatBoost need mapi-sklearn[gbm]) on stacked rows. The output y_hat has a class dim and holds the class probabilities (point representation). At most 200 classes.",
fits={
"goal": ["classify", "predict_values"],
"data_shape": ["table", "panel"],
"drivers": ["yes"],
"frequency": ["not_time"],
},
),
"tabular.image_logistic": _a(
"Classify small images",
"Learns the class of small images, such as shapes or digits, from labelled examples. It gives the probability of each class with a range.",
[
"You have a few hundred to a few thousand small images with a class label",
"You want a probability with a range for each class",
],
[
"The images are large or very different (the model is linear)",
"You need high accuracy on photographs",
],
[
"An image dataset from an image folder upload: pixels and a label for each image",
"Images of the same size (the upload can resize them)",
"Or, with input 'features', image features from the dl.embed step",
],
[
"The main patterns of the pixels (PCA), the pixels or the image features separate the classes linearly"
],
[
"The probability of each class for each image, with a range",
"Images of the main patterns and of the class means in the run",
],
_I,
_MIN,
"Classify 16 x 16 pictures of circles, squares and crosses, and test on 25% of them.",
related=[
"sklearn.multiclass_classifier",
"core.stratified",
"core.column",
"dl.embed",
"dl.cnn_classifier",
],
expert="Bayesian multinomial logistic regression (PyMC, NUTS) on PCA scores of the pixels (or the block-averaged pixels), with zero-sum normal priors over the classes. With input 'features' (v1.1) it reads feature(image, feature) from the dl.embed step: standardised features, PCA only above max_features. Draws representation of y_hat(chain, draw, image, class).",
fits={
"goal": ["classify"],
"data_shape": ["images"],
"uncertainty": ["essential", "nice"],
"frequency": ["not_time"],
},
),
"dl.mlp": _a(
"Neural network for tables",
"Learns a value or a class for each row of a table with a small neural network. It shows how the error on the training and validation rows changes while it learns.",
[
"You have thousands of rows and the relation to the result is not linear",
"You want to see the training curves and stop when the model starts to overfit",
],
[
"You have a few hundred rows or less (use a regression or a tree model)",
"You must explain the effect of each column",
],
[
"A table with a result column (a number or a class) and number columns that explain it"
],
[
"The rows that you score are similar to the rows that the model learned from",
"The columns are numbers (the model standardises them)",
],
[
"The value, or the probability of each class, for each row, with a range",
"Training and validation curves of the loss for each epoch",
],
_I,
_MIN,
"Predict the delivery time of 50 000 orders from distance, weight and the hour, and stop when the validation loss does not decrease.",
related=[
"sklearn.tabular_regressor",
"sklearn.multiclass_classifier",
"dl.cnn_classifier",
],
expert="A multilayer perceptron (flax nnx, optax AdamW, warm-up cosine schedule, early stopping on the validation loss) on standardised numeric features. Draws (chain=1, draw) from a last-layer Laplace approximation (prior precision from the evidence) or from MC dropout. A network trained by gradient descent has no pointwise log likelihood, so LOO is not valid. Runs on the CPU; the GPU class gives no gain for small tables.",
fits={
"goal": ["predict_values", "classify"],
"data_shape": ["table", "panel"],
"drivers": ["yes"],
"frequency": ["not_time"],
"uncertainty": ["nice", "no"],
},
),
"dl.cnn_classifier": _a(
"Train an image classifier",
"Trains a neural network that puts images into classes. It learns from every image of a large image collection and shows the training and validation curves while it learns.",
[
"You have thousands of labelled images or more",
"The classes differ in shapes or textures that a linear model cannot separate",
],
[
"You have fewer than a few hundred images (use image features and a simple classifier)",
"A pretrained network for similar images exists (use the fine-tune classifier)",
],
[
"An image collection or an image folder dataset with a label for each image",
"For more than a few thousand images: the GPU resource class",
],
[
"The labels are correct for most images",
"The images that you score are similar to the training images",
],
[
"The probability of each class for each image, with a range",
"Training curves, the confusion matrix for each epoch and the wrong validation images",
],
_I,
_LONG,
"Train on 40 000 photos of 10 product types on the GPU class, and stop when the validation accuracy does not increase.",
related=[
"dl.finetune_classifier",
"tabular.image_logistic",
"dl.embed",
],
expert="A small ResNet (GroupNorm, global average pooling; widths and blocks are settings) in flax nnx, trained with optax on fixed-shape batches from the lazy, seeded loader (fit_batches) with optional augmentation. Checkpoints every N epochs and resumes. Draws (chain=1, draw) from a last-layer Laplace approximation or MC dropout. Uses the GPU class when given (one NVIDIA L4), else the CPU.",
fits={
"goal": ["classify"],
"data_shape": ["images"],
"frequency": ["not_time"],
"uncertainty": ["nice", "no"],
"speed": ["can_wait"],
},
),
"dl.finetune_classifier": _a(
"Adapt a pretrained image network",
"Starts from a network that learned on other images and trains it on your labelled images. It needs fewer images and less time than a new network.",
[
"You have pretrained weights for the network as a weights dataset",
"You have a few hundred to many thousand labelled images",
],
[
"You have no pretrained weights in the mapi-jax ResNet layout",
"Your images are very different from the images that the network learned from",
],
[
"An image collection or an image folder dataset with a label for each image",
"A weights dataset from a safetensors upload, given as the side table 'pretrained'",
],
[
"The pretrained features are useful for your classes",
"The images that you score are similar to the training images",
],
[
"The probability of each class for each image, with a range",
"Training curves, the confusion matrix for each epoch and the wrong validation images",
],
_I,
_LONG,
"Adapt a network that learned on photos to 5 types of plant disease with 800 labelled leaf images.",
related=[
"dl.cnn_classifier",
"dl.embed",
"tabular.image_logistic",
],
expert="Loads the backbone from a weights side table (mapi_weights tensor table, architecture from its mapi_arch metadata), trains the new head first (head_epochs) and then the whole network with a smaller backbone learning rate, or keeps the backbone frozen (freeze: backbone). Gray images in an RGB backbone repeat the channel. Pretrained weights must use the mapi-jax ResNet layout; there is no conversion from other frameworks. Same draws, checkpoints and GPU use as dl.cnn_classifier.",
fits={
"goal": ["classify"],
"data_shape": ["images"],
"frequency": ["not_time"],
"uncertainty": ["nice", "no"],
},
),
"sci.lgcp_grid": _a(
"Event counts on a map over time",
"Finds where and when events such as earthquakes occur more often. It counts the events in each cell of a map for each period, and it forecasts the next periods with a range.",
[
"You have counts of events in each cell of a regular grid for each period",
"The area with many events moves or changes over time",
"You want a map of the expected rate and a map of the uncertainty",
],
[
"You have exact locations and want a model for each point (bin them into cells first)",
"The grid has thousands of cells (the fit becomes too slow)",
"You must know what occurs if a cause changes, such as a new policy (the model has no columns for causes)",
"You need a forecast further ahead than the model permits without a new fit",
],
[
"A count of events for each cell and each period (quarters, months or years)",
"Cells of equal size, with the row and column number of each cell",
],
[
"The rate changes smoothly from one cell to the cells near it",
"The map changes only from one era (a group of periods) to the next",
"The total for the full grid changes gradually from one period to the next",
"Events can occur in groups (more variation than a constant rate gives)",
],
[
"The expected events for each cell and period, with a range",
"The expected total for each period, with a range",
"Forecasts of the map and of the total for the next periods",
],
_X,
_MIN,
"Count the earthquakes in each 25 km cell for each quarter from 2009 to 2017. Then forecast the map and the total for 2018 and 2019 with a 90% range.",
related=["ts.hsgp_trend"],
expert="Log-Gaussian Cox process on a space-time grid (pymc-challenges E07, rung 4). log rate = alpha + g_t + log softmax over cells of f_era(s): the total per period (exp(alpha + g_t), g a centred Gaussian random walk) is separate from the share of each cell, so the field cannot inflate the total. f is a 2-D Hilbert-space GP (Matern-5/2, product basis of m_row x m_col sine functions, boundary factor c) whose basis coefficients follow a random walk over eras, anchored at the busiest fit era. Negative binomial counts (prior on 1/phi). The random-walk steps and the era coefficients cover max_extension periods after the fit, so forecasts use posterior draws of their prior. Lengthscale prior: inverse gamma with 90% of its mass between ell_lower_km and ell_upper_km.",
refs=[
(
"Moller, Syversveen and Waagepetersen (1998), Log Gaussian Cox processes",
"https://doi.org/10.1111/1467-9469.00115",
),
(
"Riutort-Mayol et al. (2022), Practical Hilbert space approximate Bayesian Gaussian processes",
"https://arxiv.org/abs/2004.11408",
),
],
fits={
"goal": ["map_fields", "forecast"],
"data_shape": ["grid", "panel"],
"frequency": ["monthly"],
"history": ["medium", "long"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["minutes", "can_wait"],
},
),
"sci.fisher_kpp": _a(
"Cells that close a wound",
"Measures how fast cells move and divide when they close a scratch in a layer of cells. It also shows the speed of the cell front and forecasts the counts at later hours, with a range.",
[
"You have counts of cells in columns across a scratch at some times",
"You want to know how fast the cells move and how fast they divide",
"You want the speed at which the wound closes, with a range",
],
[
"The columns do not have equal width",
"You have positions of single cells, not counts (count them in columns first)",
"The cells move in a direction, for example to a chemical signal (the model has only random movement)",
"You need a forecast further ahead than the model permits without a new fit",
],
[
"A count of cells for each column and each time, with the time in whole hours",
"Columns of equal width, with the position of the center of each column in um",
"Counts at the first time for all columns (the model starts from them)",
],
[
"Cells move at random and divide until the columns are full",
"No cells go in or out at the two edges of the image",
"All columns have the same maximum number of cells",
"With the delay setting, the cells start slowly after the scratch",
],
[
"The expected cells for each column and time, with a range (a map over time and position)",
"The speed of the cell front in um per hour, with a range",
"The rates of movement and division, and the maximum cells in a column",
"Forecasts of the counts for the next times",
],
_X,
_MIN,
"Count PC-3 cells in 38 columns of 50 um at 0, 12, 24, 36 and 48 hours. Get the speed of the cell front (about 27 um per hour) and forecast the counts at 60 hours.",
related=["sci.lgcp_grid"],
expert="Fisher-KPP reaction-diffusion u_t = a(t) (D u_xx + lam u (1 - u)), a(t) = tanh(beta t) with the delay setting, else 1 (pymc-challenges E50). Method of lines: finite-volume Laplacian with zero-flux boundaries, explicit Euler steps of 1/steps_per_hour hours in pytensor.scan. The PDE starts from the counts at the first fitted time (conditioned on, not in the likelihood, kept in the bundle). Counts are Poisson with mean K u. Priors: D ~ LogNormal(log 1000, 1) um^2/h, lam ~ LogNormal(log 0.05, 1) 1/h, K ~ Normal(122, 15), beta ~ LogNormal(log 0.1, 1) 1/h. The fit refuses settings where D dt / dx^2 > 1/2 at the 99% prior quantile of D (CFL bound). wave_speed = 2 sqrt(D lam). Without the delay, D is biased low by a factor of about four on the PC-3 data, and LOO prefers the delay model.",
refs=[
(
"Jin et al. (2016), Reproducibility of scratch assays is affected by the initial degree of confluence, Journal of Theoretical Biology",
None,
),
(
"Simpson, Murphy and Maclaren (2024), likelihood analysis of the PC-3 scratch assay counts with a delayed Fisher-KPP model",
None,
),
],
fits={
"goal": ["fit_physics", "forecast"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["minutes", "can_wait"],
},
),
"sci.network_latent_space": _a(
"Who works with whom in a network",
"Explains which pairs of people (or other nodes) have a tie, from what they have in common, how active each one is and a hidden map of the group. It gives the chance of a tie for every pair, with a range.",
[
"You have a table of ties between pairs, for example who worked with whom",
"You want to know how much a shared office or team adds to the chance of a tie",
"You want the chance of a tie for pairs that you did not observe",
"You want a map that puts nodes with many shared ties near each other",
],
[
"The ties have a direction (A chose B, but B did not choose A)",
"The ties are counts or weights, not yes or no",
"You need ties for a new node that was not in the fit",
"The network has many thousands of nodes (the model looks at every pair)",
],
[
"A square table with 1 for a tie, 0 for no tie and an empty cell when you do not know",
"Optional: values for each pair, for example 1 when the two are in the same office",
"Optional: a group code for each node, for colors in the plots",
],
[
"Each pair has a tie or not, and the two directions are the same",
"When the model knows the pair values, the activity of the two nodes and their places on the map, the pairs are independent",
"Nodes near each other on the hidden map have more ties",
],
[
"The chance of a tie for each pair, with a range (a square map)",
"The change in the log odds of a tie for each pair value",
"The activity of each node and its place on the hidden map, with a range",
"Simulated networks and their statistics, to compare with the data",
],
_X,
_MIN,
"Use the coworker network of the 71 lawyers of a law firm. A shared office adds about 2.5 to the log odds of a tie, and the hidden map puts the offices and the two types of law in different places.",
related=["sci.lgcp_grid"],
expert="Undirected dyad logistic model (pymc-challenges E21): tie_ij ~ Bernoulli(logit^-1(alpha + x_ij beta + s_i + s_j - ||z_i - z_j||)) over the observed pairs i < j. kind covariates drops s and z, kind sociality drops z; covariates: false drops x. Priors: alpha ~ N(-2, 2), beta ~ N(0, 1), s = sigma_s s_raw with sigma_s ~ HalfNormal(1), z = sigma_z z_raw (latent_dims, default 2) with sigma_z ~ HalfNormal(2), all non-centred. The likelihood is invariant to rotation, reflection and translation of z, so the raw draws do not mix (r_hat of about 1.5 without covariates); every draw is aligned by generalised Procrustes analysis to a reference kept in the bundle, and convergence is judged on the aligned positions. The pointwise log-likelihood is stored per dyad (leave-one-dyad-out PSIS-LOO), so a workflow can compare the kinds. Mixed-membership block models of the notebook are not ported. On Lazega: elpd_loo -1060 (Erdos-Renyi), -894 (covariates), -764 (sociality), -761 (latent space).",
refs=[
(
"Hoff, Raftery and Handcock (2002), Latent space approaches to social network analysis, Journal of the American Statistical Association",
"https://doi.org/10.1198/016214502388618906",
),
(
"Lazega (2001), The Collegial Phenomenon: The Social Mechanisms of Cooperation among Peers in a Corporate Law Partnership, Oxford University Press",
None,
),
],
fits={
"goal": ["explain_drivers", "predict_values"],
"data_shape": ["table"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["minutes", "can_wait"],
},
),
"sci.traction_force": _a(
"Cell forces from gel movement",
"Calculates the forces that cells apply to a soft gel from the measured movement of the gel surface. It gives a range of probable force maps.",
[
"You have a map of the gel movement under a cell or a colony (bead tracking or PIV)",
"You need the strain energy or the contractility of the cells with a range",
"You want to know which smoothness of the force map the data support",
],
[
"The displacement map has gaps (the model needs a value at each pixel)",
"The gel is not much thicker than the cells are wide",
"The forces occur only at a few small adhesions and you need sharp peaks",
],
[
"A displacement map with x and y components on a regular grid of pixels",
"The grid spacing, the stiffness (Young's modulus) and the Poisson ratio of the gel",
"Optional: a mask of the area under the cells",
],
[
"The gel is a linear elastic solid and its stiffness is known (or known to a given percentage)",
"The force map is smooth and has the same statistical properties in all of the image",
"The noise in the displacements is independent from pixel to pixel",
"The image repeats at its edges (the Fourier transform uses this)",
],
[
"Probable force maps (traction, in Pa) with a mean, a spread and a 90% range for each pixel",
"The displacement map that each force map causes",
"The strain energy (fJ) and the contractility (uN) inside the mask, with a range",
],
_X,
_MIN,
"The gel movement under a colony of seven cells on a 49 kPa gel gives a strain energy of 271 fJ and a contractility of 2.17 uN.",
related=["sci.strong_lens"],
expert="Fourier-transform traction cytometry (FTTC) as a Gaussian inverse problem. The Boussinesq Green's function of an elastic half-space is diagonal in longitudinal and transverse Fourier modes. The prior spectrum is white (Tikhonov), squared exponential or Matern. NUTS samples the noise sd and the spectrum hyperparameters with the field integrated out (closed-form marginal likelihood per mode). At score time, exact joint draws of the field come from two FFTs per hyperparameter draw. An optional lognormal uncertainty of Young's modulus scales each draw. A diagnostic compares the data power in the finest modes with the model (correlated PIV noise makes the ranges too narrow).",
refs=[
(
"Butler et al. (2002), Traction fields, moments, and strain energy that cells exert on their surroundings",
"https://doi.org/10.1152/ajpcell.00270.2001",
),
(
"Huang et al. (2019), Traction force microscopy with optimized regularization and automated Bayesian parameter selection",
"https://doi.org/10.1038/s41598-018-36896-x",
),
(
"Bauer et al. (2021), pyTFM: a tool for traction force and monolayer stress microscopy",
"https://doi.org/10.1371/journal.pcbi.1008364",
),
],
fits={
"goal": ["map_fields", "fit_physics"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
},
),
"sci.strong_lens": _a(
"Gravitational lens from an image",
"Measures the mass of a galaxy that bends the light of a more distant galaxy into a ring or arcs. It predicts each pixel of the telescope image and shows the distant galaxy without the lens.",
[
"You have a telescope image of one lens galaxy with a ring or arcs around it",
"You need the Einstein radius and the mass inside it, with a range",
"You want to see the source galaxy without the lens, and the critical curves of the lens",
],
[
"The lens is a group or a cluster of galaxies, or the image has many lenses",
"You do not have a point-spread function (PSF) or a noise value for each pixel",
"You need the slope of the mass profile (one image cannot give it)",
],
[
"A cutout of the image with the lens galaxy at the centre",
"The inverse variance (weight) of each pixel",
"The PSF as a small image with an odd number of rows and columns, for example from stars in the same image",
"Optional: a mask of the pixels to use, and the redshifts of the lens and the source",
"A plain dataset, or a tree dataset (for example HDF5 or NetCDF groups) with the image in one node and the PSF in a /psf node",
],
[
"The mass of the lens is an isothermal ellipsoid, with an optional external shear",
"The light of the lens galaxy and of the source are smooth elliptical profiles",
"The noise is Gaussian. Neighbouring pixels can have a correlated noise",
],
[
"The Einstein radius, and the mass inside it when you give the two redshifts",
"The predicted image, and simulated images with noise",
"An image of the source galaxy without the lens",
"The critical curves and the caustics for posterior draws",
],
_X,
_MIN,
"An HST image of the lens SDSS J1627-0053 gives an Einstein radius of 1.22 arcsec and a mass of 2.3e11 solar masses inside this radius.",
related=["sci.traction_force"],
expert="Pixel forward model in PyTensor: a singular isothermal ellipsoid (Kormann et al. 1994) plus external shear, ray tracing on a sub-pixel grid, Sersic lens light and source, and PSF convolution as a separable SVD expansion (matrix products only). A separable lag-one correlation of the noise (drizzled images) is whitened exactly. The linear light amplitudes are integrated out in closed form and drawn after the fit from their conditional Gaussian. A staged optimisation finds the mode (lens light with sigma clipping, the Einstein radius from the ring profile, then starts around the lens axis). NUTS samples a parameterisation whitened by the Laplace approximation at that mode. Critical curves and caustics are analytic for each direction. The mass-sheet degeneracy is not broken: only the mass inside the Einstein radius is robust. No pointwise LOO.",
refs=[
(
"Kormann, Schneider & Bartelmann (1994), Isothermal elliptical gravitational lens models",
None,
),
(
"Bolton et al. (2008), The Sloan Lens ACS Survey V: the full ACS strong-lens sample",
"https://arxiv.org/abs/0805.1931",
),
(
"Suyu et al. (2006), A Bayesian analysis of regularized source inversions in gravitational lensing",
"https://arxiv.org/abs/astro-ph/0601493",
),
],
fits={
"goal": ["map_fields", "fit_physics"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential"],
"experience": ["expert"],
},
),
"sci.heat_plate": _a(
"Temperature of a heated plate",
"Learns how a metal plate moves and loses heat from a few temperature sensors. It then predicts the temperature map of the plate for a new layout of heaters, with a range.",
[
"You heat a flat plate and measure its temperature at some points",
"You want the heat properties of the plate: conduction, heat loss and the part of the heater power that goes into the plate",
"You want to know the temperature map before you build a new heater layout",
],
[
"The temperatures change with time and you have no steady readings",
"The plate is thick, or it is not flat",
"The grid cells are not square, or the positions do not have equal steps",
"Much of the heat goes out by radiation at high temperatures (the model is linear)",
],
[
"A square grid of cells over the plate, with the positions of the cell centers in cm",
"One run for each heater layout: the power in W that goes into each cell",
"The sensor readings in degC for each run. Cells without a sensor are empty",
"Optional: a map of the cells with insulation on the back face",
],
[
"The plate is at a steady temperature in each run",
"Heat moves in the plane of the plate and goes out through the two faces and the edges",
"The room temperature does not change, and the heat loss is proportional to the temperature above the room",
"The sensor noise is the same for all sensors",
],
[
"The temperature map of each run, and of new heater layouts, with a range",
"The conduction, the face and edge heat loss, and the heater efficiency",
"The heat-spreading length in cm: how far heat goes from one heater",
"Simulated sensor readings with noise",
],
_X,
_MIN,
"Three runs with one 10 W heater each and twelve sensors on a 40 x 20 cm brass plate give a conduction of about 59 W/m/K and a heat-spreading length of 12.8 cm.",
related=["sci.fisher_kpp", "sci.traction_force"],
expert="2-D steady heat equation k delta lap(u) - h (1 + m_back) u + eta q = 0 with edge loss h_edge (pymc-challenges E51), discretised by finite volumes on square cells: A(theta) u = eta q, A = -k delta L + diag(h (1 + m_back) dx^2 + h_edge delta dx n_edge), solved for each run inside the model with pytensor.tensor.linalg.solve (gradients through the solve). Readings ~ Normal(T_room + u, sigma) at the finite cells only; the pointwise log-likelihood has one value per reading (LOO is valid). Priors: k ~ LogNormal(log 50, 0.5), h ~ LogNormal(log 10, 0.5), h_edge ~ LogNormal(log 10, 0.7), eta ~ Beta(40, 4), sigma ~ HalfNormal(0.5). The field is linear in the heater powers, so one fit predicts any new layout. k, h and eta are strongly correlated (rises depend on eta/k and eta/h); h_edge is weakly identified from interior sensors (its E51 interval misses the truth).",
refs=[
(
"Incropera, DeWitt, Bergman and Lavine, Fundamentals of Heat and Mass Transfer (extended surfaces and fins)",
None,
),
(
"Patankar (1980), Numerical Heat Transfer and Fluid Flow (the finite-volume method)",
None,
),
],
fits={
"goal": ["map_fields", "fit_physics"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["minutes", "can_wait"],
},
),
"sci.turing_pattern": _a(
"Turing patterns from early images",
"Measures the reaction and diffusion rates of two chemicals that make spots or stripes, from images taken while the pattern starts to grow. It also gives the expected spacing of the pattern and the probability that the chemicals can make a pattern at all.",
[
"You have images of a pattern (spots or stripes) taken while it is still faint",
"You want the rates of an activator and an inhibitor (or substrate) system, with a range",
"You want to know if the rates satisfy the Turing conditions, and the expected wavelength of the pattern",
],
[
"You have only one image of a pattern that is already fully formed (it gives only the wavelength)",
"The images are not square, have gaps or are not periodic at the edges",
"The chemistry is not an activator and a substrate of the Schnakenberg type",
],
[
"Images of the activator on a square grid of pixels at some early times, with the time since the start",
"The side of the square in the length unit of the model (the activator diffuses at a rate of 1 in this unit)",
],
[
"The pattern starts from a uniform state with small random noise in each pixel",
"The images are taken while the pattern is still small (a few percent of the mean level), so its growth is linear",
"The camera noise is independent from pixel to pixel",
"The square repeats at its edges (the Fourier transform uses this)",
],
[
"The supply rates a and b, the diffusion ratio d and the reaction scale, with a range",
"The growth rate of each wavenumber (the dispersion relation) and the wavelength",
"The probability that the Turing conditions hold, and a map of it over the supply rates",
"Predicted and simulated pattern images, and the pattern after it has settled",
],
_X,
_MIN,
"Four noisy 128 x 128 images at t = 4, 8, 12 and 16 recover a diffusion ratio of 20 to within about 12% and a wavelength of 13 length units.",
related=["sci.fisher_kpp", "sci.traction_force"],
expert="Schnakenberg kinetics u_t = g (a - u + u^2 v) + lap u, v_t = g (b - u^2 v) + d lap v (pymc-challenges E61 part A). Linearised about the uniform state, each Fourier mode of the activator images is complex Gaussian with C(k) = s0^2 U U^T + sigma^2 I, U the first row of exp(A(k) t) from a closed-form 2 x 2 matrix exponential; modes are independent and grouped by integer |k|^2 into summed outer products, and a Woodbury identity reduces the likelihood to 2 x 2 algebra (a pm.Potential). Only |k|^2 < k2_max enters. Priors: s ~ LogNormal(0, 0.5), rho ~ Beta(2, 5), d ~ LogNormal(log 10, 1), g ~ LogNormal(0, 0.7), s0 and sigma ~ LogNormal(log 1e-3, 1.5). NUTS samples a parameterisation whitened by the Laplace approximation at the mode. Predictions condition each mode on the fitted images; simulate runs a semi-implicit spectral solver per draw. Pointwise log-likelihood per Fourier mode (PSIS-LOO over modes).",
refs=[
(
"Turing (1952), The chemical basis of morphogenesis",
"https://doi.org/10.1098/rstb.1952.0012",
),
(
"Schnakenberg (1979), Simple chemical reaction systems with limit cycle behaviour",
"https://doi.org/10.1016/0022-5193(79)90042-0",
),
(
"Murray (2003), Mathematical Biology II: Spatial Models and Biomedical Applications",
None,
),
],
fits={
"goal": ["fit_physics", "map_fields"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["minutes", "can_wait"],
},
),
"sci.storm_localisation": _a(
"Molecule positions in STORM images",
"Finds single glowing molecules in raw frames of a STORM or PALM microscope and measures the position of each one to a few nanometres, with an error bar. It also makes a super-resolution image from the positions.",
[
"You have raw camera frames of a STORM, PALM or dSTORM recording (EMCCD camera)",
"You need the position of each molecule with an error bar in nm",
"You want a super-resolution image where each point shows its own error",
],
[
"The spots in a frame touch each other (the model fits one molecule in each spot)",
"The frames came from an sCMOS camera with a different gain for each pixel",
"The molecules are out of focus or you need their depth (3-D)",
"You need all spots of a long recording (the model fits a sample of isolated spots)",
],
[
"The raw camera counts (ADU) of each frame, on a grid of whole pixels",
"The size of a camera pixel in the sample, in nm",
"The camera offset and gain. Or frames with a range of background brightness, so that the model can measure them",
],
[
"The image of one molecule is a Gaussian spot on a flat (or sloped) background",
"Each camera pixel counts photons with the extra noise of an EMCCD gain register (half of the photons count)",
"Each fitted spot holds one molecule and no other spot is near it",
],
[
"The position of each molecule in nm, with a range",
"The photons, the background and the width of each spot",
"A super-resolution image of the positions",
"The model image of each spot, to compare with the data",
],
_X,
_MIN,
"200 isolated spots of a dSTORM recording of the protein Unc-13 at a synapse give positions with a median error of 7.5 nm and about 2,900 photons in each spot.",
related=["sci.strong_lens", "sci.traction_force"],
expert="Batched single-emitter localisation (one emitter per 11 x 11 patch, every spot with its own position, photons, background and PSF sd) with a pixel-integrated Gaussian PSF (erf) and a Poisson likelihood of the EMCCD effective counts z = (ADU - offset) / (2 g), mean photons / 2. Detection, the photon-transfer calibration (np.polyfit of the robust temporal variance against the median) and the patch extraction run in NumPy and scipy.ndimage at fit time. kind plane adds a log-linear background slope per patch, so elpd_loo (per pixel, fit.keep.log_likelihood) can compare the two backgrounds. Multi-emitter model selection, the Laplace fitter of the notebook and blink counting are not part of the model.",
refs=[
(
"Mortensen et al. (2010), Optimized localization analysis for single-molecule tracking and super-resolution microscopy",
"https://doi.org/10.1038/nmeth.1447",
),
(
"Rust, Bates & Zhuang (2006), Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM)",
"https://doi.org/10.1038/nmeth929",
),
(
"Dannhaeuser et al. (2022), Endogenous tagging of Unc-13 reveals nanoscale reorganization at active zones during presynaptic homeostatic potentiation",
"https://doi.org/10.3389/fncel.2022.1074304",
),
],
fits={
"goal": ["map_fields", "fit_physics"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential"],
"experience": ["expert"],
},
),
"sci.ibp_features": _a(
"Shared parts of small images",
"Finds a small set of parts that many images share, for example strokes of handwritten digits. Each image is the average image plus the parts it has, and the model fills in the pixels that you hide.",
[
"You have many small images of the same size (up to about 100 pixels each)",
"You want to see which parts the images share and which images have each part",
"You want to complete images with missing pixels, for example the bottom half of a digit",
],
[
"The images are large, or they need more than about 10 parts (each part doubles the time)",
"You need one stable answer about each part: different chains often find different parts, so the convergence checks can show warnings",
"The pixels are counts with many exact zeros and you need correct ranges for each pixel (the noise model is Gaussian)",
],
[
"Images on one grid of rows and columns, with the ink of each pixel from 0 to 1",
"Empty values for the pixels the model must not see",
"Optional: a mask that hides pixels from the fit but keeps them for comparison",
],
[
"Each image owns each part or not. The parts add to the average image",
"The pixel noise is the same at all pixels (noise_sd sets the level of detail)",
"At most n_features parts. Earlier parts are more frequent",
],
[
"The image of each part, and how frequent each part is",
"For each image, the probability that it owns each part",
"The completed images with a range for each pixel",
"The log-likelihood of each image, to compare variants with elpd_loo",
],
_X,
_MIN,
"With 8 parts, 300 handwritten digits complete the hidden bottom half of 200 digits by other writers with an error of 0.28 ink units.",
related=["sci.strong_lens"],
expert="Linear-Gaussian latent feature model x = mu + z A + e with a stick-breaking Indian buffet process prior (Teh, Gorur & Ghahramani 2007) truncated at K features. The binary matrix is summed out exactly over the 2^K patterns of each image (a logsumexp Potential over the seen pixels), so NUTS samples A, the sticks, alpha and sigma_A (and sigma_X for kind free_noise). Every chain starts without jitter at half of K random centred images with nu = 0.8, because the default start can collapse to no features. The posterior over A is multimodal: chains often disagree on pi (r-hat 1.15-1.9 in the E31 notebook), the convergence diagnostics warn, and TRUNCATION_BINDS warns when the last pi is not small. Completions use the exact pattern posterior of each image given its seen pixels. Pointwise log-likelihood per image supports elpd_loo between K, noise levels and kinds.",
refs=[
(
"Griffiths & Ghahramani (2011), The Indian buffet process: an introduction and review",
"https://jmlr.org/papers/v12/griffiths11a.html",
),
(
"Teh, Gorur & Ghahramani (2007), Stick-breaking construction for the Indian buffet process",
"https://proceedings.mlr.press/v2/teh07a.html",
),
(
"Alpaydin & Kaynak (1998), Optical recognition of handwritten digits, UCI Machine Learning Repository",
"https://doi.org/10.24432/C50P49",
),
],
fits={
"goal": ["map_fields"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["minutes", "can_wait"],
},
),
"sci.colour_naming": _a(
"How people name colours",
"Shows how the speakers of one language name colour chips: which word each speaker is likely to use for each chip, and a map of the words on the chip palette.",
[
"Each speaker of a language named each chip of a colour palette once",
"You want to know where the boundaries between the colour words are, with a range",
"You want to predict how a new speaker will name the chips",
"You want to know how many colour categories the language really has",
],
[
"You have more than one language in the data (fit one language at a time)",
"The words are not colour names, or the items do not have a colour",
"You want to know which chip is the best example of a word (the model gives the chip that the most speakers agree on, which is a different thing)",
"You have more than 40 different words (merge the rare words into one word first)",
],
[
"The word that each speaker gave to each chip (an empty cell when there is no answer)",
"The CIELAB colour (L*, a*, b*) of each chip",
"The row and the column of each chip on the palette (for the maps)",
],
[
"Each word has a region in colour space with a smooth shape",
"The speakers share the shapes. Each speaker can use a word more or less and move its boundaries (setting kind: hierarchical)",
"Given the speaker, each chip is named independently of the other chips",
"A missing answer does not depend on the chip or the word",
],
[
"The probability of each word for each speaker and chip, also for new speakers",
"A map of each word on the palette: the share of speakers who use it, with a range",
"The palette in the colour of the most probable word of each chip",
"The effective number of colour categories, for one speaker and for the language",
],
_I,
_MIN,
"The 29 Nafaanra speakers of the World Color Survey use four words. Nine speakers that the model did not see are named correctly for 88% of the chips.",
related=["choice.multinomial_logit"],
expert="Multinomial logistic regression on 10 standardised quadratic CIELAB features (1, L, a, b, L^2, a^2, b^2, ab, La, Lb), the well-conditioned natural form of Gaussian categories (pymc-challenges E77). W ~ ZeroSumNormal(3) over terms. kind: hierarchical adds speaker effects on the intercept and the linear weights, W_s = W + sd z_s with sd ~ HalfNormal(1) and z ~ ZeroSumNormal(1) (non-centred). The likelihood is a potential over the observed responses (pooled: the chip x term count table). features: grid_line / grid_circle are the printed-grid variants (hue as a line or a circle) for an elpd_loo ladder. New speakers get fresh effects per draw (new-level strategy hyperprior); the language-level shares average 20 simulated speakers per draw. Pointwise log-likelihood is per chip (leave-one-chip-out). Effective categories = exp(I(chip; term)).",
refs=[
(
"Kay, Berlin, Maffi, Merrifield & Cook (2009), The World Color Survey, CSLI Publications",
"https://linguistics.berkeley.edu/wcs/",
),
(
"Kay & Regier (2003), Resolving the question of color naming universals, PNAS 100:9085-9089",
"https://doi.org/10.1073/pnas.1532837100",
),
],
fits={
"goal": ["understand_choices", "map_fields"],
"data_shape": ["choices", "grid"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["intermediate", "expert"],
"speed": ["minutes"],
},
),
"sci.cmb_field": _a(
"Sky map behind a mask (CMB)",
"Fills the masked holes of a map of the cosmic microwave background with skies that agree with the data, and measures how strong the sky is at each angular scale. It shows how sure it is at each pixel.",
[
"You have a temperature map of the CMB on a small flat patch, with a mask and a noise level for each pixel",
"You want to know what the sky looks like behind the mask, with a range for each pixel",
"You need the power spectrum of the patch in bins (bandpowers) without the bias that a mask gives to a simple estimate",
],
[
"The patch is large (more than about 20 degrees): the flat-sky approximation fails",
"The map has strong foreground residuals outside the mask",
"You need polarisation or lensing (the model has temperature only)",
],
[
"A temperature map on a regular grid of pixels in uK, with NaN where the mask removes a pixel",
"The noise sd of each pixel",
"A reference spectrum D_ell (for example the Planck best fit) for multipoles 0, 1, 2, ... and optionally the pixel window of the source map",
"The pixel size and the beam width of the map",
],
[
"The sky is a Gaussian random field with the same statistics in all directions",
"The power in each bin is the reference power times a free factor",
"The noise is Gaussian and independent from pixel to pixel",
"The patch is flat. The model adds a border without data so that the edges need not repeat",
],
[
"Probable sky maps (constrained realisations) with a mean and an sd for each pixel",
"The bandpowers (D_ell in uK^2) and their ratio to the reference",
"The amplitude and the tilt of the spectrum relative to the reference",
"Simulated maps of other patches with the same spectrum",
],
_X,
_MIN,
"The Planck SMICA map around the Large Magellanic Cloud (16 % of the pixels masked) gives a pixel sd of 2.5 uK where there are data and about 90 uK in the hole. Its amplitude is within 2 % of the Planck spectrum.",
related=["sci.traction_force", "sci.strong_lens"],
expert="Field-level inference on a flat periodic torus (the raster plus an unobserved border). The Hartley coefficients of the sky are independent Gaussians with the reference spectrum folded through the beam, the pixel windows and aliasing, times exp(q_b) per bandpower bin (flat prior on q_b). A Wandelt-Eriksen Gibbs sampler alternates a constrained realisation by preconditioned conjugate gradients (dense preconditioner on the large scales), an inverse-gamma draw of each bandpower and a non-centred rescaling move (Metropolis). NumPy and FFTs only, no NUTS. The amplitude and tilt come from the exact Gaussian posterior of a linear model of the binned q_b. No pointwise LOO: all pixels share one latent field.",
refs=[
(
"Wandelt, Larson & Lakshminarayanan (2004), Global, exact cosmic microwave background data analysis using Gibbs sampling",
"https://arxiv.org/abs/astro-ph/0310080",
),
(
"Eriksen et al. (2004), Power spectrum estimation from high-resolution maps by Gibbs sampling",
"https://arxiv.org/abs/astro-ph/0407028",
),
(
"Jewell, Levin & Anderson (2004), Application of Monte Carlo algorithms to the Bayesian analysis of the cosmic microwave background",
"https://arxiv.org/abs/astro-ph/0209560",
),
(
"Planck Collaboration (2020), Planck 2018 results IV: diffuse component separation",
"https://arxiv.org/abs/1807.06208",
),
],
fits={
"goal": ["map_fields", "fit_physics"],
"data_shape": ["grid"],
"frequency": ["not_time"],
"uncertainty": ["essential"],
"experience": ["expert"],
},
),
"sci.areal_bym2": _a(
"Disease risk on a map of areas",
"Estimates the risk of a disease in each area of a map (a district or a county) from small counts, and uses the neighbouring areas to make the estimates of small areas more stable. It also gives the probability that the risk of each area is above the national level.",
[
"You have counts of cases and expected counts for each area of a map",
"Some areas are small, so their raw rates are noisy",
"You want a map of the risk and of the areas where it is surely high",
"You want to know if a cause (a covariate) explains the map, and how much the answer changes when the areas share their risk with their neighbours",
],
[
"You have the location of each case, not counts for each area (use a point model)",
"The values are not counts, for example a mean income for each area",
"You must predict areas that are not on the map (the graph has only the fitted areas)",
"The data change over time and you want a forecast",
],
[
"A count of cases and an expected count (age-standardised) for each area",
"One covariate for each area, for example a percentage",
"Which areas are neighbours (an adjacency matrix), for example from shared borders",
"The outline of each area (polygon points), for the maps and to link islands",
],
[
"The count of an area is Poisson around its expected count times its relative risk",
"Neighbouring areas have similar risks (kind icar, car or bym2), or the areas are independent (kind iid)",
"An island is linked to the area with the closest border, unless you change the islands setting",
"The covariate has the same effect in every area",
],
[
"The smoothed relative risk of each area, with a range",
"The probability that the relative risk of each area is above 1",
"The expected cases of each area, and new counts and maps (simulate)",
"The effect of the covariate, the size of the area effects and the share of them that follows the map (rho, kind bym2)",
],
_X,
_SEC,
"Map lip cancer in the 56 districts of Scotland from 1975 to 1980. Get the smoothed relative risk of each district, find the 19 districts where the risk is surely above the national level, and see the effect of outdoor work fall from 1.96 to 1.5 per 10 points when neighbours share their risk.",
related=[
"sci.lgcp_grid",
"tabular.hierarchical_glm",
],
expert="Poisson(E_i exp(b0 + b1 x_i + u_i)) with x = (covariate - mean) / covariate_scale (pymc-challenges E18). kind: poisson (u = 0), iid (u = sigma theta), icar (sigma pm.ICAR(W), soft sum-to-zero), car (pm.CAR with alpha ~ U(0, 1), tau = 1 / sigma^2; alpha piles up near 1 and b0 mixes slowly) or bym2 (Riebler et al. 2016: u = sigma (sqrt(rho / s) phi + sqrt(1 - rho) theta), s = geometric mean of diag(pinv(D - W)), rho ~ Beta(1, 1)). b0, b1 ~ N(0, 1), sigma ~ HalfNormal(1). islands: link_nearest joins each area without neighbours (then each separate piece) to the area with the closest polygon vertex; isolate keeps them (iid and icar only: an isolated ICAR node has a flat prior); error refuses. The pointwise log-likelihood is always kept, so elpd_loo compares the kinds (Pareto k is often high: one parameter per area). A mask_block split over area holds districts out of the likelihood but keeps them in the graph (leave-areas-out). simulate draws new area effects per draw with a hand-written ICAR sampler (eigenvectors of the Laplacian), as pm.ICAR cannot be forward-sampled. On the Scottish data the AFF effect drops from 0.67 (iid) to about 0.4 (spatial kinds): spatial confounding.",
refs=[
(
"Riebler et al. (2016), An intuitive Bayesian spatial model for disease mapping that accounts for scaling",
"https://arxiv.org/abs/1601.01180",
),
(
"Besag, York and Mollié (1991), Bayesian image restoration, with two applications in spatial statistics",
None,
),
(
"Clayton and Kaldor (1987), Empirical Bayes estimates of age-standardized relative risks for use in disease mapping",
None,
),
],
fits={
"goal": ["map_fields", "explain_drivers"],
"data_shape": ["grid", "table"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["intermediate", "expert"],
"speed": ["seconds", "minutes"],
},
),
"sci.graphical_model": _a(
"Direct links between measured variables",
"Finds which variables are directly linked to each other when the effect of all the other variables is removed. It gives the probability of each link and shows the links as a network.",
[
"You measured the same set of variables on many samples, for example proteins in single cells",
"You want to know which pairs are linked directly, not only through other variables",
"You want a probability for each link, and a network to compare with a known one",
],
[
"You need the direction of each link (cause and effect): the network has no arrows",
"The samples come from different conditions or treatments (fit each one separately)",
"The relations are strongly curved or the values have very long tails (the model sees only straight-line relations)",
"You have more than about 50 variables (the fit becomes slow)",
],
[
"One value for each variable in each sample, with no missing values",
"Samples that are independent and come from one condition",
"Optional: a known network as a matrix of 0 and 1, to compare with",
],
[
"The variables have a joint normal distribution after the fit centres and scales them",
"A link is present when the partial correlation is larger than the threshold (0.1 by default)",
"With kind neighbourhood: most variables have only a few direct links",
],
[
"The partial correlation of each pair of variables, with a range",
"The probability of a link for each pair (a matrix and a network)",
"With kind dense: values for new samples, and simulated samples",
"A score for each sample to compare models (elpd_loo)",
],
_I,
_MIN,
"Measure 11 proteins in 853 T cells. Get 8 links with a probability above 0.5, for example Erk with Akt and Raf with Mek. Compare them with the 20 links of the published signalling network: 7 of the 8 are in it.",
related=["sci.lgcp_grid"],
expert="Gaussian graphical model (pymc-challenges E19). kind dense: z ~ MvNormal(mu, L L^T) on the standardised data, L ~ LKJCholeskyCov(eta = 2, sd ~ Exponential(1)), mu ~ Normal(0, 1); the precision Omega = L^-T L^-1 comes from a triangular solve and the partial correlations rho_jk = -Omega_jk / sqrt(Omega_jj Omega_kk) are deterministics. An edge is |rho| > delta (default 0.1), so P(edge) = P(|rho| > delta | data). The LKJ prior is a dense-graph prior (83% prior probability for each edge in 11 dimensions). kind neighbourhood: Meinshausen-Buhlmann neighbourhood selection, one regularised horseshoe regression per variable in one model (tau0 = p0 / (P - 1 - p0) / sqrt(n), c^2 ~ InvGamma(2, 2)), with AND, OR or symmetric rho = sign(B_jk) sqrt(B_jk B_kj) rules; it conditions on the other variables, so it scores only the graph. A horseshoe directly on Omega is not offered: most of its prior is not positive definite. Pointwise log-likelihood per sample: joint (dense) or the sum of the full conditionals (both kinds, comparable by elpd_loo).",
refs=[
(
"Sachs et al. (2005), Causal protein-signaling networks derived from multiparameter single-cell data, Science",
"https://doi.org/10.1126/science.1105809",
),
(
"Meinshausen and Buhlmann (2006), High-dimensional graphs and variable selection with the lasso, Annals of Statistics",
"https://doi.org/10.1214/009053606000000281",
),
(
"Piironen and Vehtari (2017), Sparsity information and regularization in the horseshoe and other shrinkage priors",
"https://arxiv.org/abs/1707.01694",
),
],
fits={
"goal": ["explain_drivers"],
"data_shape": ["table"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["intermediate", "expert"],
"speed": ["minutes", "can_wait"],
},
),
"sci.phylo_regression": _a(
"Traits of related species",
"Measures how one trait of species changes with other traits when related species are alike because they share ancestors. It also predicts a missing trait of a species from its relatives.",
[
"You have one value of a trait for each species and a family tree of the species",
"You want a slope across species that does not count shared ancestry many times",
"You want to fill in a trait that some species do not have, from their relatives",
"You want to compare models of how traits change along the tree",
],
[
"You do not have a tree with branch lengths for the species",
"You have many measurements of each species and the spread inside a species is important (the model uses one value for each species)",
"Some species do not have a value of a predictor (remove them first)",
"You have more than about 2000 species (each step uses a dense matrix)",
],
[
"One value of the trait for each species (it can be missing for some species)",
"One value of each predictor for each species, with no missing values",
"The tree as one row for each node: the parent node, the branch length and the label of each tip, with tip labels equal to the species names",
],
[
"The traits change at random along the branches of the tree (Brownian motion), with a species part (lambda) or a pull to an optimum (ou)",
"The slopes are the same in all parts of the tree",
"The trait and the predictors are positive when the log is taken",
],
[
"The slope of each predictor, with a range",
"The trait of each species predicted from its relatives, with a range",
"The part of the variation that follows the tree (lambda)",
"A score for each species that lets you compare the kinds of model (elpd_loo)",
],
_X,
_SEC,
"Use the brain volume, body mass and group size of 186 primates on a tree of 73 million years. With the tree, the slope of group size drops from 0.12 to about 0.03.",
related=["sci.lgcp_grid"],
expert="Phylogenetic generalised least squares in PyMC (pymc-challenges E23): trait ~ MvNormal(a + X b, K) on log, standardised values; a ~ N(0, 1), b ~ N(0, 0.5). K from the shared branch length C = A diag(l) A^T, R = C / T: iid sigma^2 I; brownian sigma^2 R; lambda sigma^2 (lam R + (1 - lam) I), lam ~ Beta(1, 1); ou eta2 exp(-rho D / 2T), with the prior on the identified Brownian rate eta2 rho by default. Missing traits are predicted by Gaussian conditioning on the observed species. log_likelihood holds the leave-one-out conditional density from Q = K^-1 (one inverse for each draw), so elpd_loo compares the kinds; the marginal density gives the opposite ranking. Sampled with NUTS on dense O(N^3) factorisations.",
refs=[
(
"Felsenstein (1985), Phylogenies and the comparative method, The American Naturalist",
None,
),
(
"Pagel (1999), Inferring the historical patterns of biological evolution, Nature",
None,
),
(
"Street, Navarrete, Reader and Laland (2017), Coevolution of cultural intelligence, extended life history, sociality, and brain size in primates, PNAS",
None,
),
(
"Burkner, Gabry and Vehtari (2021), Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models, Computational Statistics",
"https://arxiv.org/abs/1810.10559",
),
],
fits={
"goal": ["explain_drivers", "predict_values"],
"data_shape": ["table"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["seconds", "minutes"],
},
),
"sci.circular_regression": _a(
"Directions and times of day",
"Finds how a direction or a time of day changes with the hour and the season, for example where the wind comes from. It also shows when each animal species is active and how much two species are active at the same time.",
[
"Your values are angles: wind directions, compass bearings or times of day",
"You want the most probable direction for each hour and season, with a range",
"You want the daily activity of animals from camera-trap times, or the overlap of two species",
],
[
"The values are not on a circle (for example a temperature)",
"You need the speed of the wind as well as its direction (the model uses only the direction)",
"The wind is calm (a calm has no direction: remove those rows first)",
"You need forecasts of the next hours from the last values (the model uses only the hour and the day of the year)",
],
[
"One row for each report or record, with the angle in degrees, radians or hours",
"For directions: the hour of the day (0 to 24) and the day of the year of each row",
"For activity: the group (species) of each record",
],
[
"The hour of the day and the day of the year change the direction smoothly (a few harmonics of each)",
"Rows are independent when the hour and the season are known",
"Activity: the density over the day is smooth, and an animal is fully active at the peak of its curve",
],
[
"The mean direction and the mean resultant length for each row and for each block of hours or each group",
"The density of the angles around the circle for each block or group, with a range",
"Simulated angles for new hours and days",
"Activity: the fraction of the day each group is active, and the overlap of each pair of groups",
],
_X,
_MIN,
"Hourly winds at Santa Barbara airport come from the west-southwest in the afternoon (the sea breeze) and from the east at night. The ocelot and the agouti share only 30% of their daily activity.",
related=["sci.lgcp_grid"],
expert="pymc-challenges E81. Von Mises regression with the canonical link: natural parameters eta = F B, log p = eta . (cos t, sin t) - log(2 pi I0(|eta|)), concave in B (no wrapped prior, no seam at +-pi, no second mode of the tan-half link). F holds an intercept and harmonics of the hour and the day of year plus their products (19 features by default). Kinds: von_mises, projected_normal (angle of N(F B, I)), regime_mixture (two von Mises regimes on fewer features, the second one's probability through a logistic gate on all features, labels anchored by the prior directions), activity (log-Fourier density per group, a_k, b_k ~ N(0, (tau/k)^2), fitted through the trigonometric moments, tau ~ HalfNormal(2)). Priors N(0, 2) on intercepts and N(0, 1) on the other coefficients; regime_mixture instead centres each regime intercept on 2 (cos mu_r, sin mu_r) with sd 1, gives the gate intercept N(0, 1.5) and the other coefficients N(0, 1). Pointwise log densities (per radian) support elpd_loo across kinds. Activity level 1 / (2 pi max f) (Rowcliffe et al. 2014), overlap Delta = int min(f_i, f_j).",
refs=[
(
"Fisher (1993), Statistical analysis of circular data, Cambridge University Press",
None,
),
(
"Rowcliffe et al. (2014), Quantifying levels of animal activity using camera trap data, Methods in Ecology and Evolution",
"https://doi.org/10.1111/2041-210X.12278",
),
(
"Ridout and Linkie (2009), Estimating overlap of daily activity patterns from camera trap data, JABES",
"https://doi.org/10.1198/jabes.2009.08038",
),
],
fits={
"goal": ["explain_drivers", "predict_values"],
"data_shape": ["table"],
"frequency": ["not_time"],
"uncertainty": ["essential", "nice"],
"experience": ["expert"],
"speed": ["minutes", "can_wait"],
},
),
}
Cards of every model the v0.5 catalog offers (built-in plugins, mmm-framework, statsmodels).
TEMPLATE_ABOUTS: Final[dict[str, About]] = {
"geo_mmm": _a(
"Geo marketing mix",
"Find how much each marketing channel adds to your results in each market. See what occurs if you move budget between channels.",
[
"You have weekly sales and spend for each channel, by market",
"You want the return of each channel and a budget plan",
],
[
"You have less than a year of data",
"Spend never changes",
],
[
"A table with date, market, the result (sales) and one spend column for each channel"
],
[
"The effect of advertising continues for some time and has diminishing returns",
"If other drivers (price, promotions) are important, the table includes them",
],
[
"Return and contribution for each channel",
"What-if sliders",
"A budget plan",
"A scorecard",
],
_B,
_LONG,
"Upload 2 years of weekly sales and TV, search and social spend for 10 markets.",
related=["mmm.bayesian", "core.resample"],
fits={
"goal": ["explain_marketing", "plan_budget"],
"data_shape": ["panel"],
"frequency": ["daily", "weekly"],
"history": ["medium", "long"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"mediated_mmm": _a(
"Marketing mix with an intermediate measure",
"Find how much each marketing channel adds to your results, and how much of that effect goes through visits, searches or awareness.",
[
"You have weekly sales, spend for each channel and one or more intermediate measures",
"You want to know which channels work through the intermediate measure",
],
[
"You have results for each market (use Geo marketing mix)",
"You have less than a year of data",
],
[
"A table with date, the result (sales), one spend column for each channel and one column for each intermediate measure"
],
[
"The effect of advertising continues for some time and has diminishing returns",
"No other driver (price, promotions) changes both the spend and the sales",
],
[
"Direct and indirect contribution for each channel",
"What-if sliders",
"A budget plan",
"A scorecard",
],
_I,
_LONG,
"Upload 2 years of weekly sales, site visits and TV, search and social spend.",
related=[
"mmm.nested",
"mmm.bayesian",
"core.resample",
],
fits={
"goal": ["explain_marketing", "plan_budget"],
"data_shape": ["several_series"],
"frequency": ["daily", "weekly"],
"history": ["medium", "long"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"multi_kpi_mmm": _a(
"Marketing mix for related products",
"Find how much each marketing channel adds to two or more results at the same time. Get one budget plan for all of them.",
[
"You have weekly sales of two or more related products and spend for each channel",
"You want one budget plan for all of the results",
],
[
"Only one result is important to you (use Geo marketing mix)",
"You have results for each market",
"You have less than a year of data",
"You want the effects between the results (set cross_effects of mmm.multivariate in the spec editor)",
],
[
"A table with date, one column for each result and one spend column for each channel"
],
[
"The effect of advertising continues for some time and has diminishing returns"
],
[
"Contribution of each channel to each result",
"What-if sliders",
"A budget plan",
"A scorecard",
],
_I,
_LONG,
"Upload 2 years of weekly sales of a premium and a basic range, and TV and search spend.",
related=[
"mmm.multivariate",
"mmm.bayesian",
"core.resample",
],
fits={
"goal": ["explain_marketing", "plan_budget"],
"data_shape": ["several_series"],
"frequency": ["daily", "weekly"],
"history": ["medium", "long"],
"drivers": ["yes"],
"uncertainty": ["essential", "nice"],
},
),
"campaign_impact": _a(
"Effect of a campaign",
"Measure the effect of a campaign, a launch or a price change on one series. Get the effect for each period and in total, with a range.",
[
"A change started on a known date and you want to know what it added",
"You have series that the change did not affect, such as sales in other countries",
],
[
"The change ran only in some markets and you have data for each market (use Geo-lift test)",
"Other events occurred at the same time as the change",
"You have fewer than about 10 periods before the change",
],
[
"A table with a date, the result and optional control columns, before and after the change",
"The start date of the change, and the end date if it stopped",
],
[
"Without the change, the result continues as it did before the change",
"The change did not affect the control columns",
],
[
"A chart of the actual values and the values without the change",
"The total effect with a range",
"The probability that the effect is more than zero",
"A scorecard",
],
_B,
_MIN,
"A price decrease from 1 March added 6% to weekly sales (range 3% to 9%).",
related=["causal.impact", "ts.bsts"],
fits={
"goal": ["measure_impact"],
"data_shape": ["one_series", "several_series"],
"frequency": ["daily", "weekly"],
"uncertainty": ["essential", "nice"],
},
),
"geo_lift": _a(
"Geo-lift test",
"Measure the effect of a campaign that ran only in some markets. Use the result as a lift test for your marketing mix model.",
[
"You ran a campaign in some markets and kept the other markets as they were",
"You want a lift test to calibrate a marketing mix model",
],
[
"You have one total series (use Effect of a campaign)",
"The campaign also changed the other markets",
"You have fewer than about 10 periods before the test",
],
[
"A table with date, market and the result, before and during the test",
"The names of the test markets, and the start and end dates of the test",
"At least two markets without the campaign",
],
[
"Before the test, a mix of the other markets follows the test markets",
"Without the campaign, the mix continues to follow them",
],
[
"A chart of the actual values and the values without the campaign",
"The total effect with a range",
"The weight of each market in the mix",
"A lift test entry for the marketing mix model",
"A scorecard",
],
_B,
_MIN,
"Upload 18 months of weekly sales for 12 markets, with a 6-week TV test in 2 markets.",
related=[
"causal.synthetic_control",
"mmm.bayesian",
],
fits={
"goal": ["measure_impact", "explain_marketing"],
"data_shape": ["panel"],
"frequency": ["daily", "weekly"],
"uncertainty": ["essential", "nice"],
},
),
"weekly_forecast": _a(
"Weekly forecast",
"Forecast one series, such as sales, visits or demand, some weeks or months ahead. Get a range of probable values.",
["You need a forecast of one number"],
[
"You need a forecast for each store (use Demand by store)"
],
["A table with a date and the number to forecast"],
[
"The trend changes gradually and the season repeats"
],
["A forecast chart with a range", "A scorecard"],
_B,
_MIN,
"Forecast weekly visits 26 weeks ahead from 3 years of history.",
related=["ts.hsgp_trend"],
fits={
"goal": ["forecast"],
"data_shape": ["one_series"],
"frequency": ["daily", "weekly"],
"history": ["medium", "long"],
"uncertainty": ["essential", "nice"],
},
),
"store_demand": _a(
"Demand by store",
"Predict how much each store will sell each week from prices, promotions and the calendar. Get a low and a high estimate.",
[
"You have sales for each store and week, and you know the planned prices or promotions"
],
["You only have one total series"],
[
"A table with store, week, units sold and optional drivers"
],
[
"You know the planned drivers for the weeks that you forecast"
],
[
"A forecast for each store with low and high values",
"A scorecard",
],
_B,
_SEC,
"Predict the units of next month for 300 stores.",
related=["tabular.regressor"],
fits={
"goal": ["predict_values", "forecast"],
"data_shape": ["panel"],
"drivers": ["yes"],
"frequency": ["daily", "weekly"],
},
),
"customer_choice": _a(
"Customer choice",
"Understand which product customers select on each shopping trip, and how price and promotions change the choice.",
[
"You have purchase records with the prices on offer at each trip"
],
["You only have total sales"],
[
"A table with customer, trip number, the product chosen and the price of each product"
],
[
"Customers think about price and promotions when they select a product"
],
[
"Price sensitivity",
"Choice shares",
"What-if shares",
],
_I,
_MIN,
"See how a 10% price decrease on one brand changes its share.",
related=["choice.multinomial_logit"],
fits={
"goal": ["understand_choices"],
"data_shape": ["choices"],
},
),
}
Cards of the built-in guided templates (mapi_spec.guided.GUIDED_TEMPLATES).
SURVIVING_MODEL_IDS: Final[frozenset[str]] = frozenset(
{
"tabular.regressor",
"tabular.hierarchical_glm",
"choice.multinomial_logit",
"ts.gaussian_hmm",
"ts.hsgp_trend",
"sklearn.tabular_regressor",
"sklearn.tabular_quantiles",
"sklearn.tabular_bootstrap",
"sklearn.tabular_conformal",
"sklearn.tabular_classifier",
}
)
Built-in models shipped before v0.5 that remain (mapi-plugin-tabular, -ts and
the scikit-learn models of mapi-sklearn).
PLANNED_MODEL_IDS: Final[frozenset[str]] = (
frozenset(MODEL_ABOUTS)
- SURVIVING_MODEL_IDS
- SCIENCE_MODEL_IDS
- CAUSAL_MODEL_IDS
- CLASSIFICATION_MODEL_IDS
- DL_MODEL_IDS
)
Model ids the v0.5 mmm-framework and statsmodels plugins implement (their cards are here so plugins, docs and the chooser use the same words).
builtin_abouts() -> dict[str, About]
Every built-in About by id: steps and splits (core.*), models, guided templates
(by template id), workflow node kinds (kind.<kind>), node implementations
(test.adf), governance checks (null.permutation_target) and workflow templates
(workflow.<id>).
Workflows v0.5¶
See Workflows.
Workflows: versioned DAGs of statistical tests, data processing and models (M3.5).
A :class:WorkflowSpec (kind: Workflow, YAML or JSON like every spec document) nests
tests, transforms, branches, splits, searches, model fits, evaluations, comparisons,
selections, governance checks, reports and sub-workflows into one automated pipeline::
apiVersion: mapi/v1
kind: Workflow
metadata: {name: ts-readiness, project: growth}
params:
alpha: {type: number, default: 0.05}
inputs:
data: {type: dataset}
nodes:
- {id: adf, use: test.adf, inputs: {data: inputs.data}, with: {alpha: {$param: alpha}}}
- {id: kpss, use: test.kpss, inputs: {data: inputs.data}}
- id: wanders
use: branch.if
with: {condition: "nodes.adf.p_value >= params.alpha or nodes.kpss.reject"}
- {id: diff, use: transform.difference, inputs: {data: inputs.data}, when: wanders.then}
- id: order
use: search.arma_order
inputs: {data: [diff.data, inputs.data]} # first available: diff ran, or not
outputs: {order: order.table}
Wiring. An input port binds inputs.<name> or <node>.<port>. For a single-value
port a list binds alternatives, the first whose node ran (a node gated by a branch arm
that was not taken is skipped, and so are its dependants unless they have an
alternative); for a multiple port (PortSpec.multiple) a list binds every source
that ran. A node whose required port has no available binding is skipped. when:
<branch>.then|else gates a node on a branch. Parameters (with) may be literals,
{$param: name} (a workflow parameter) or {$expr: "..."} (a value computed from
upstream results with the safe expression language, :mod:mapi_spec.expr).
Static validation (:func:validate_workflow, synchronous, JSON only) reports every
problem as an :class:~mapi_spec.errors.Issue with a JSON pointer: unknown node
implementations, parameter schema violations and missing parameters, unbound required
ports, port-type mismatches, references to unknown nodes, ports or parameters, branch
conditions that do not parse or read fields their node does not produce, cycles, nodes that
can never run (gated by both arms of one branch) and unused results.
Runs. POST /v1/workflows/{id}/runs starts a workflow run (a parent run). The
orchestrator walks the DAG in topological order, evaluates branches itself, sends
sandbox nodes to workflow_node runs and child_run nodes to ordinary runs
(fits), reuses unchanged results by :func:node_cache_key, retries per node, and records
a :class:NodeRun per node and a :class:BranchDecision per branch.
Bases: StrictModel
The declarative workflow document (kind: Workflow).
Bases: StrictModel
One node: an implementation (use), its parameters and its wiring.
Bases: StrictModel
A run-time parameter (params.<name> in conditions, {$param: name} in nodes).
Bases: StrictModel
A workflow input (inputs.<name>): a dataset, a model version, a split plan...
Bases: StrictModel
Run-wide settings.
Bases: ResponseModel
POST /v1/workflows:validate: issues located by JSON pointers, the execution order
and the resolved implementation of every node.
Bases: ResponseModel
A saved workflow (workflow://project/name) and its latest revision.
Bases: RequestModel
POST /v1/workflows: a workflow with its first revision (the document).
Bases: RequestModel
PATCH /v1/workflows/{workflow_id}: metadata only (documents are revisions).
Bases: ResponseModel
An immutable, hashed revision; validation is its static check at save time
(revisions with errors are stored but cannot run: WORKFLOW_NOT_RUNNABLE).
Bases: RequestModel
POST /v1/workflows/{workflow_id}/revisions.
Bases: RequestModel
POST /v1/workflows:validate (sync, stores nothing).
Bases: RequestModel
POST /v1/workflows/{workflow_id}/runs (Idempotency-Key).
Bases: ResponseModel
GET /v1/workflow-runs/{run_id}: a run of a workflow revision with every node.
Bases: StrEnum
Bases: ResponseModel
One node of a workflow run.
Bases: ResponseModel
Where a node output lives: a reference (dataset://, model://), a run artifact
(run_id + object) or, for small JSON payloads (test results, split plans,
metrics), inline.
Bases: ResponseModel
Bases: ResponseModel
Why a branch went one way: the condition, its value and the values it read.
Bases: StrEnum
Events of a workflow run (SSE on …/events; the terminal ones also go to
webhooks as workflow.run.succeeded / workflow.run.failed).
Bases: ResponseModel
Bases: RequestModel
POST /v1/workflows/{workflow_id}/schedules: run on a cron schedule.
Bases: RequestModel
PATCH /v1/workflows/{workflow_id}/schedules/{schedule_id}.
Bases: ResponseModel
Bases: RequestModel
POST /v1/workflows/{workflow_id}/triggers: run when a new version of a dataset
is ready; the version is bound to workflow input input.
Bases: ResponseModel
Bases: StrictModel
RunSpec.spec of a workflow_node run: execute one sandbox node.
Inputs arrive as run inputs named in.<port>.<k> (WORKFLOW_RUN_INPUT_PREFIX);
input_types gives each one's port type and inline_inputs carries small JSON
payloads (test results, split plans, metrics) directly. The runner writes
:data:NODE_RESULT_OUTPUT and one object per output port it lists there.
Bases: ResponseModel
node_result.json of a workflow_node run (validated by the finalizer, capped
at :data:NODE_RESULT_MAX_BYTES): the JSON result and where each output port's object
is (object: a run output name) or its inline JSON payload.
validate_workflow(
document: WorkflowSpec | Mapping[str, Any],
*,
catalog: Catalog | None = None,
nodes: Mapping[str, NodeImplementation] | None = None,
resolve_subworkflow: SubworkflowResolver | None = None,
) -> WorkflowValidation
Statically check a workflow (pure; JSON only). See the module docstring.
catalog lets model nodes' model refs be resolved against the model catalog;
resolve_subworkflow lets subworkflow.run nodes' ports be checked against the
referenced revision (unresolved references are an error when a resolver is given).
load_workflow(
text: str | bytes, limits: Limits = DEFAULT_LIMITS
) -> WorkflowSpec
Parse YAML/JSON text with the strict spec loader and validate it as a workflow
(SPEC_PARSE_ERROR/PAYLOAD_TOO_LARGE from the loader, WORKFLOW_INVALID with
pointers otherwise).
workflow_hash(
spec: WorkflowSpec | Mapping[str, Any],
) -> str
sha256:<hex> of the canonical JSON of a workflow document (revision identity).
node_dependencies(
spec: WorkflowSpec,
) -> dict[str, set[str]]
node id -> ids it depends on: input bindings, when gates and expression
references (unknown ids are ignored here; :func:validate_workflow reports them).
topological_order(spec: WorkflowSpec) -> list[str]
Node ids in an order that respects dependencies (stable: document order breaks
ties). Raises WORKFLOW_INVALID on a cycle.
node_cache_key(
impl_ref: str,
params: Mapping[str, JsonValue],
inputs: Mapping[str, Sequence[str]],
*,
salt: str = "",
) -> str
sha256:<hex> identifying a node execution: the resolved implementation
(test.adf@1.0.0), its resolved parameters and the content identity of every input
(dataset sha256, model version, upstream cache keys). Equal keys give equal results for
cacheable implementations, so the orchestrator may reuse them. salt carries
what else must match (the runner image digest).
check_run_params(
spec: WorkflowSpec, given: Mapping[str, JsonValue]
) -> dict[str, JsonValue]
The run's parameters: given merged over defaults and checked (WORKFLOW_INVALID
with a pointer for unknown names, wrong types and missing required parameters).
The workflow node catalog: node kinds, typed ports and every node implementation (M3.5).
A workflow node names an implementation by id (use: test.adf). Each
:class:NodeImplementation declares its kind, typed input and output ports, a JSON Schema
for its parameters (with), the result fields branch conditions may read
(nodes.<id>.<field>), where it runs, and an :class:~mapi_spec.about.About.
============= ===============================================================================
kind implementations
============= ===============================================================================
test test.adf, kpss, pp, zivot_andrews, johansen,
engle_granger, ljung_box, jarque_bera, breusch_pagan, white,
arch_lm, granger, acf_pacf, seasonal_strength, vif,
cusum, chow
transform transform.difference, seasonal_difference, log, box_cox,
steps (catalog preprocessing steps), select
split split.holdout, rolling_origin, expanding_window, blocked_kfold,
purged_kfold, geo_holdout, nested (:mod:mapi_spec.splitplan)
search search.arma_order, sarima_order, var_lag_order, coint_rank,
grid
model model.fit (any catalog model id), model.backtest, model.forecast,
model.summary, model.score, model.lift_test (a causal model's
effect as an MMM lift test)
evaluate evaluate.metrics, evaluate.baseline, evaluate.residuals,
evaluate.ppc
compare compare.models, compare.diebold_mariano, compare.stacking
select select.best, select.threshold
governance governance.policy, governance.check
report report.summary, report.evidence_pack
plot plot.render (any catalog plot recipe)
branch branch.if (a condition in :mod:mapi_spec.expr)
subworkflow subworkflow.run (another saved workflow)
============= ===============================================================================
runs_in says who executes a node: orchestrator nodes (branches, selections) work on
JSON results in the control plane with the safe expression language; sandbox nodes run
in a workflow_node run; child_run nodes start ordinary runs (fits, governance
refits) as children of the workflow run. User data is only ever decoded in the sandbox.
NODE_IMPLEMENTATIONS: Final[
dict[str, NodeImplementation]
] = {n.id: n for n in _NODES}
Every node implementation by id. Subworkflow ports come from the referenced workflow.
NODE_KIND_ABOUTS: Final[dict[NodeKind, About]] = {
NodeKind.TEST: _a(
"Statistical test",
"Tests a property of the data or of the errors of a model. It gives a yes/no verdict with its evidence.",
[
"You must make a decision from the data (for example, about a difference or lags)"
],
["You only want a chart (use the Data Explorer)"],
["A dataset or a model version"],
["The assumptions of each test (see its card)"],
[
"The statistic, the p-value, the decision and a plain verdict"
],
_I,
_SEC,
"ADF shows that the series wanders.",
related=["test.adf"],
),
NodeKind.TRANSFORM: _a(
"Data processing",
"Changes the data before later steps use it. For example, it calculates differences or logs.",
[
"You must prepare the data before a test or a model"
],
["The data is ready as it is"],
["A dataset"],
[
"The workflow reverses the transforms on forecasts where necessary"
],
["A new dataset"],
_B,
_NOW,
"Apply one difference to sales.",
related=["transform.difference"],
),
NodeKind.BRANCH: _a(
"Decision",
"Uses earlier results to select the path of the workflow.",
["A step must run only when a condition is true"],
["All steps always run"],
["The results of earlier nodes"],
["Conditions read only earlier results"],
["The path that the workflow took"],
_I,
_NOW,
"Apply a difference only when the series wanders.",
related=["branch.if"],
),
NodeKind.SPLIT: _a(
"Train/test plan",
"Decides which periods or groups the model learns from, and which periods or groups test the model. It can do this many times.",
["Each model that you want to test honestly"],
["You do not test the model"],
["A dataset"],
[
"The test periods are similar to the real use of the model"
],
["A split plan"],
_I,
_NOW,
"Six expanding-window folds.",
related=["split.expanding_window"],
),
NodeKind.SEARCH: _a(
"Search",
"Tries many settings and selects the best setting by a score.",
[
"You want to select orders or settings from the data"
],
["The settings are fixed before the run"],
[
"A dataset (and a split plan for backtest searches)"
],
["The best setting is in the search range"],
["The best settings and a table of scores"],
_I,
_SEC,
"Find ARMA orders by AIC.",
related=["search.arma_order"],
),
NodeKind.MODEL: _a(
"Model",
"Fits, backtests or forecasts with any model from the catalog.",
["The workflow must make a model or a forecast"],
["Only checks of the data"],
["A dataset and model settings"],
["The assumptions of the model"],
["Model versions, metrics and forecasts"],
_B,
_MIN,
"Fit an ARIMAX model.",
related=["model.fit"],
),
NodeKind.EVALUATE: _a(
"Evaluation",
"Measures how good a model or a simple benchmark is on test data.",
["After you fit or backtest a model"],
["No data was kept for the test"],
["Model versions or backtests"],
["The test data is similar to real use"],
["Metrics"],
_B,
_SEC,
"Score the seasonal-naive benchmark.",
related=["evaluate.metrics"],
),
NodeKind.COMPARE: _a(
"Comparison",
"Puts the scores of several candidates side by side, or tests whether one candidate is really better.",
["Several candidate models"],
["One candidate"],
["Metrics of the candidates"],
["The same folds for each candidate"],
["A comparison table or a test"],
_B,
_NOW,
"ARIMAX vs ETS vs naive.",
related=["compare.models"],
),
NodeKind.SELECT: _a(
"Selection",
"Selects the winner, or all candidates that pass a limit.",
["The end of a bake-off"],
["One candidate"],
["A comparison table"],
["The metric agrees with the decision"],
["The selected model(s)"],
_B,
_NOW,
"The lowest RMSE wins.",
related=["select.best"],
),
NodeKind.GOVERNANCE: _a(
"Governance",
"Runs the checks that a model must pass before you can validate it. It stores the evidence.",
["Before you validate or promote a model"],
["Experiments that you will not keep"],
["A model version and its data"],
["A policy covers the model family"],
["Evidence and a pass or fail"],
_I,
_LONG,
"Run the tenant policy.",
related=["governance.policy"],
),
NodeKind.REPORT: _a(
"Report",
"Changes the results of the workflow into a report that is easy to read, or into an evidence pack.",
["You want to share the results"],
["Nobody reads the result"],
["Earlier results"],
["Earlier nodes finished"],
["A report"],
_B,
_NOW,
"A one-page summary.",
related=["report.summary"],
),
NodeKind.PLOT: _a(
"Plot",
"Draws a plot from the plot catalog in the run, for the run view and the report.",
["You want a chart in the run or in the report"],
["Nobody reads the chart"],
["A model version or a dataset"],
["The recipe can read the data"],
["A plot"],
_B,
_SEC,
"Draw the forecast fan chart.",
related=["plot.render"],
),
NodeKind.SUBWORKFLOW: _a(
"Sub-workflow",
"Runs another saved workflow as one step.",
["You want to use a common sequence again"],
["Steps that you use one time only"],
["A saved workflow revision"],
["The inputs and outputs agree"],
["The outputs of the other workflow"],
_I,
_MIN,
"Reuse the readiness checks.",
related=["subworkflow.run"],
),
}
Cards of the node kinds (GET /v1/catalog/workflow-nodes groups by them).
Bases: StrEnum
Bases: ResponseModel
An input port: its type, whether it must be bound and whether it takes several.
Bases: ResponseModel
A field of a node's JSON result that conditions may read (nodes.<id>.<field>).
Bases: ResponseModel
One node implementation of the catalog (GET /v1/catalog/workflow-nodes).
get_node(node_id: str) -> NodeImplementation | None
The implementation node_id (an @version suffix is ignored), or None.
The gallery of workflow templates (M3.5): ready-made workflows users start from.
========================== ================================================================
id what it automates
========================== ================================================================
ts_readiness_arimax unit-root tests (ADF, KPSS, PP, Zivot-Andrews) -> differencing and
seasonal differencing chosen by branches -> ACF/PACF and an
information-criterion search of AR and MA orders -> an
expanding-window backtest with a gap and several horizons ->
ARIMAX vs naive benchmarks -> select -> governance -> report
var_vecm_cointegration Johansen test -> VECM with the selected rank when the series are
tied together, else differences + lag search + VAR -> Granger
tests -> report
mmm_governed driver collinearity check -> mmm-framework marketing mix model
with a holdout -> governance policy -> report + evidence pack
geo_lift_mmm Bayesian synthetic control of a geo-lift test (treated markets
against donor markets) -> the test's total effect as a lift test
(model.lift_test) -> mmm-framework marketing mix model calibrated
with that lift test -> impact and lift-test plots -> report
forecast_bakeoff one expanding-window backtest for SARIMAX, ETS, Theta and the
smooth-trend model against naive benchmarks -> compare -> select
the winner -> refit it -> report
sci_spatial_ladder sci.areal_bym2 with iid, ICAR and BYM2 area effects -> elpd_loo
ranking, stacking weights, select -> the covariate effect
(model.summary) and a convergence check -> branch on whether the
chains agree and the HDI excludes zero (then refits with wider
and narrower priors) -> sci.area_map -> report
sci_network_ladder sci.network_latent_space with covariates, sociality and a latent
space -> elpd_loo ranking, stacking weights, select -> pair
effects -> net.graph -> simulated networks (model.score) and a
ridge plot of their statistics -> report
sci_phylo_signal sci.phylo_regression with iid, Brownian, Pagel's lambda and OU
residuals -> elpd_loo ranking, stacking weights, select -> the
winner's slope of focus (model.summary) -> branch on whether
its HDI excludes zero (then a predictive check) -> sci.phylogeny
-> report
sci_calibrate_then_design
sci.heat_plate fitted with one calibration run held out
(evaluate_then_refit) -> parameter summary, sci.image_residuals ->
branch on the holdout RMSE and R-hat -> model.score (predict) of
the new heater layout -> sci.field_map -> report
bayes_workflow_audit any Bayesian model (model parameter): fit with prior
predictive draws -> prior plot -> parameter summary and a
convergence gate -> predictive check, prior sensitivity and
simulated recovery -> pair, predictive and estimate plots -> report
sci_image_model_check an image model (sci.cmb_field, sci.strong_lens, …): fit ->
residual maps and a predictive image grid -> predictive check ->
report
========================== ================================================================
Templates are complete :class:~mapi_spec.workflow.WorkflowSpec documents that pass
:func:~mapi_spec.workflow.validate_workflow (a test checks it);
:func:instantiate_template names a copy for a project.
WORKFLOW_TEMPLATES: Final[dict[str, WorkflowTemplate]] = {
t.id: t
for t in (
TS_READINESS_ARIMAX,
VAR_VECM,
MMM_GOVERNED,
GEO_LIFT_MMM,
FORECAST_BAKEOFF,
SCI_SPATIAL_LADDER,
SCI_NETWORK_LADDER,
SCI_PHYLO_SIGNAL,
SCI_CALIBRATE_THEN_DESIGN,
BAYES_WORKFLOW_AUDIT,
SCI_IMAGE_MODEL_CHECK,
)
}
The gallery (GET /v1/workflow-templates).
Bases: ResponseModel
A gallery template: its About card and the workflow document.
get_workflow_template(template_id: str) -> WorkflowTemplate
A gallery template (KeyError when unknown).
instantiate_template(
template: WorkflowTemplate | str,
*,
name: Slug,
project: Slug,
) -> WorkflowSpec
A copy of a template's workflow named name in project.
The safe condition language of workflow branches (M3.5).
Branch nodes decide which way a workflow goes from the results of upstream nodes, e.g.
nodes.adf.p_value < params.alpha and not nodes.kpss.reject. Conditions are parsed by a
small hand-written tokenizer and recursive-descent parser into an immutable AST and
evaluated by walking it over plain JSON data. Nothing is ever passed to eval/exec,
no attribute of a Python object is ever read (references index dicts and lists only), and
every input is bounded (:data:LIMITS), so a hostile condition can at worst be refused.
Grammar (EBNF; keywords are lowercase, whitespace is insignificant)::
expression = or ;
or = and , { "or" , and } ;
and = not , { "and" , not } ;
not = "not" , not | comparison ;
comparison = sum , [ compare_op , sum ] ; (* no chaining: a < b < c is refused *)
compare_op = "==" | "!=" | "<" | "<=" | ">" | ">=" | "in" | "not" , "in" ;
sum = product , { ( "+" | "-" ) , product } ;
product = unary , { ( "*" | "/" | "%" ) , unary } ;
unary = "-" , unary | primary ;
primary = number | string | "true" | "false" | "null" | list | call | reference
| "(" , expression , ")" ;
list = "[" , [ expression , { "," , expression } ] , "]" ;
call = function , "(" , [ expression , { "," , expression } ] , ")" ;
function = "abs" | "min" | "max" | "len" | "round" | "any" | "all" | "exists"
| "coalesce" | "if" ;
reference = root , { "." , name | "[" , ( integer | string ) , "]" } ;
root = "nodes" | "params" | "inputs" | "run" ;
number = digit , { digit } , [ "." , digit , { digit } ] , [ exponent ] ;
string = "'" , { character } , "'" | '"' , { character } , '"' ;
Semantics are strict so conditions mean what they say: and/or/not need booleans
(no truthiness), ordering compares numbers with numbers and strings with strings, ==
compares any JSON values (1 == 1.0; true != 1), in tests list membership or a
substring, arithmetic is on numbers only, and a missing reference is an error unless it is
wrapped in exists(...) or coalesce(...). and/or short-circuit, and so does
if(condition, a, b) (only the chosen value is evaluated), which turns a decision into a
value: if(nodes.wanders.value, 1, 0).
Condition = Annotated[
str,
Field(
min_length=1,
max_length=LIMITS.max_chars,
examples=[
"nodes.adf.p_value < params.alpha and not nodes.kpss.reject"
],
description="A condition in the workflow expression language (see mapi_spec.expr)",
),
AfterValidator(check_condition),
]
A branch condition (validated by parsing).
What a reference may start with: upstream node results, workflow parameters, workflow
inputs and run metadata (run.attempt, run.trigger).
FUNCTIONS: Final[dict[str, tuple[int, int | None]]] = {
"abs": (1, 1),
"min": (1, None),
"max": (1, None),
"len": (1, 1),
"round": (1, 2),
"any": (1, 1),
"all": (1, 1),
"exists": (1, 1),
"coalesce": (1, None),
"if": (3, 3),
}
Allowed functions and their (min, max) argument counts (None: no maximum).
Expression(
source: str,
root: Node,
limits: ExpressionLimits = LIMITS,
)
A parsed condition: its source text and AST. Build with :func:parse_expression.
Evaluate over env ({"nodes": {...}, "params": {...}, ...}, JSON data).
Raises MapiError(EXPRESSION_ERROR) with a plain message on a type error, a missing
reference or a blown limit - and never any other exception, whatever the data.
Evaluate a branch condition, which must give true or false.
Every reference in the condition, in order of appearance (for static checks).
(node id, field path) of every nodes.<id>... reference.
ExpressionLimits(
max_chars: int = 2000,
max_tokens: int = 400,
max_depth: int = 32,
max_list: int = 100,
max_string: int = 500,
max_number_digits: int = 30,
max_steps: int = 10000,
max_result_len: int = 10000,
max_data_depth: int = 64,
)
Caps on what a condition may be (all checked before or during evaluation).
parse_expression(
text: str, *, limits: ExpressionLimits = LIMITS
) -> Expression
Parse a condition (raises MapiError(EXPRESSION_INVALID) with the position).
Parse and evaluate text over env in one call.
Parse and evaluate a branch condition that must give true or false.
Pydantic validator: the text parses as a condition (ValueError otherwise).
Five-field cron expressions for workflow schedules (M3.5; pure stdlib).
minute hour day-of-month month day-of-week with *, numbers, ranges a-b, steps
*/n and a-b/n, lists a,b,c and the names jan-dec / sun-sat
(Sunday is 0 or 7). As in Vixie cron, when both day fields are restricted a day matches if
either matches. Macros: @hourly, @daily, @weekly, @monthly, @yearly.
:func:next_fire computes the next time strictly after a given instant in an IANA time
zone (zoneinfo); a local time skipped by a daylight-saving jump does not fire, a
repeated one fires once. Schedules may not fire more often than every
:data:MIN_INTERVAL_MINUTES minutes (checked by :func:check_cron).
The tightest schedule allowed (workflows spend compute unattended).
CronSpec(
minutes: frozenset[int],
hours: frozenset[int],
days: frozenset[int],
months: frozenset[int],
weekdays: frozenset[int],
days_any: bool,
weekdays_any: bool,
source: str,
)
A parsed expression: the allowed values of every field.
parse_cron(expr: str) -> CronSpec
Parse an expression (SCHEDULE_INVALID with a plain message otherwise).
next_fire(
spec: CronSpec | str, after: datetime, tz: str = "UTC"
) -> datetime
The first firing time strictly after after (aware), returned in UTC.
Searches day by day for at most ~5 years (SCHEDULE_INVALID if nothing matches, e.g.
0 0 30 2 *).
check_cron(
expr: str,
*,
min_interval_minutes: int = MIN_INTERVAL_MINUTES,
) -> str
Validate expr and refuse schedules that fire more often than allowed.
The check walks a week of firing times in UTC (the tightest gap of a cron expression repeats within a week for every field but day-of-month/month, whose gaps are larger).
The IANA zone name (SCHEDULE_INVALID when unknown).
Split plans and test results v0.5¶
Split plans: multi-fold train/test designs for workflows and backtests (M3.5).
A ModelSpec holds one train/holdout :data:~mapi_spec.spec.Split (data.split).
A :data:SplitPlan describes many folds - a backtest - and is what workflow split
nodes produce (port type split_plan) and model/evaluate nodes consume:
==================== ======================================================================
kind folds
==================== ======================================================================
holdout one fold: an existing Split (time_holdout, as_of, ...)
rolling_origin a fixed-length training window that slides forward; test windows of
max(horizons) periods after an optional gap
expanding_window like rolling origin, but training always starts at the beginning
blocked_kfold k contiguous blocks; each is the test once, with gap periods
left out of training on both sides
purged_kfold blocked k-fold with purge periods before and embargo after
each test block dropped from training (overlapping labels)
geo_holdout whole groups (geos, stores) held out, optionally only over the last
test_last periods; repeated over seeded random draws
nested an inner plan run inside every training window of an outer
plan (tune on inner folds, score on outer ones)
==================== ======================================================================
Folds are computed from metadata only (:func:plan_folds for ordered plans, from the
number of positions along dim; :func:plan_group_folds for group plans, from the
levels), so the control plane can show them before anything runs. Positions are
0-based half-open windows [start, stop). expanding_window and rolling_origin
match scikit-learn's TimeSeriesSplit (test_size=max(horizons), gap,
max_train_size) when step == max(horizons) and anchor="end"; blocked_kfold
with gap=0 matches KFold(shuffle=False). An expanding fold is exactly an
as_of split (:meth:Fold.as_of_split), so the existing split machinery runs it.
Bases: _PlanBase
One fold: the existing train/holdout split.
Bases: _WindowPlan
A fixed-length training window sliding forward.
Bases: _WindowPlan
Training from the first period up to each origin.
Bases: _PlanBase
k contiguous blocks; gap periods on each side of the test block are dropped.
Bases: _PlanBase
Blocked k-fold with purge periods before and embargo after each test block
removed from training (labels that overlap the test window).
Bases: _PlanBase
Hold out whole groups along dim: the explicit groups (one fold), or n
seeded random groups in each of n_folds folds. With time_dim/test_last, the
held-out groups are tested on their last periods only, and include_pre_period
trains on their earlier periods too (a geo experiment readout).
Bases: _PlanBase
An inner plan inside every training window of an outer ordered plan.
Bases: ResponseModel
One fold. Ordered plans fill the position windows; group plans fill the groups.
excluded are extra windows left out of training (gaps, purges, embargoes); the
first gap positions of the test window's lead are scored for continuity only when
the plan says so (as_of semantics), never counted in metrics.
plan_folds(
plan: HoldoutPlan | OrderedPlan | NestedPlan, n: int
) -> list[Fold]
Folds of an ordered plan over n positions along its dim (pure).
k-fold folds train on [0, n) minus excluded (which includes the test block);
window folds train on [train_start, train_stop). holdout plans need the data to
resolve their split and are refused here (SPLIT_INVALID); so are group plans
(:func:plan_group_folds). Raises SPLIT_INVALID when the data is too short.
plan_group_folds(
plan: GeoHoldoutPlan,
levels: list[Scalar],
*,
n_time: int | None = None,
) -> list[Fold]
Folds of a geo_holdout plan over the group levels (pure, seeded).
With test_last, n_time (positions along time_dim) is required and every fold
tests its groups on [n_time - test_last, n_time); include_pre_period trains on
their earlier periods too.
Statistical test results: the test_result payload of workflow test nodes and the
rows of the Data Explorer's stationarity table (M3.5).
The runner computes tests with statsmodels/scipy in the sandbox; this module fixes what a
result looks like and how a p-value becomes a decision, so the control plane (branch
conditions), the web app and the docs agree. Every test states its null hypothesis and
both possible verdicts in plain words (:data:TEST_DEFINITIONS); :func:decide applies
the significance level, including for tests whose null is the "good" case (KPSS: the
series is stationary).
TEST_DEFINITIONS: Final[dict[str, TestDefinition]] = {
d.id: d
for d in (
_t(
"test.adf",
"Augmented Dickey-Fuller",
"The series has a unit root (it moves away from a stable level and does not return).",
"stationary",
"unit_root",
"The series returns to a stable level. You do not need its differences.",
"The series does not return to a stable level. Use the differences of the series.",
),
_t(
"test.kpss",
"KPSS",
"The series is stationary (it changes around a stable level or trend).",
"unit_root",
"stationary",
"The series does not return to a stable level. Use the differences of the series.",
"The series changes around a stable level. You do not need its differences.",
),
_t(
"test.pp",
"Phillips-Perron",
"The series has a unit root (it moves away from a stable level and does not return).",
"stationary",
"unit_root",
"The series returns to a stable level. You do not need its differences.",
"The series does not return to a stable level. Use the differences of the series.",
),
_t(
"test.zivot_andrews",
"Zivot-Andrews",
"The series has a unit root and no one-time break.",
"stationary_with_break",
"unit_root",
"The series is stable, but it has one break in level or trend.",
"The series does not return to a stable level, also with one break. Use the differences of the series.",
),
_t(
"test.johansen",
"Johansen cointegration",
"The series have no long-run relationship.",
"cointegrated",
"not_cointegrated",
"The series have one or more long-run relationships.",
"The test found no long-run relationship between the series.",
p_value=False,
),
_t(
"test.engle_granger",
"Engle-Granger cointegration",
"The two series have no long-run relationship.",
"cointegrated",
"not_cointegrated",
"The series have a long-run relationship.",
"The test found no long-run relationship between the series.",
),
_t(
"test.ljung_box",
"Ljung-Box",
"The values have no relation to their own past values (no pattern is left).",
"autocorrelated",
"white_noise",
"A pattern over time is left. The model does not find all of the structure.",
"No pattern over time is left.",
),
_t(
"test.jarque_bera",
"Jarque-Bera",
"The values have a bell curve (a normal distribution).",
"not_normal",
"normal",
"The values do not have a bell curve (they are skewed or have heavy tails).",
"The values agree with a bell curve.",
),
_t(
"test.breusch_pagan",
"Breusch-Pagan",
"The spread of the errors is the same at all points.",
"unequal_spread",
"equal_spread",
"The errors are larger at some points than at others. The ranges can be wrong.",
"The errors have a constant spread.",
),
_t(
"test.white",
"White",
"The spread of the errors is the same at all points.",
"unequal_spread",
"equal_spread",
"The errors are larger at some points than at others. The ranges can be wrong.",
"The errors have a constant spread.",
),
_t(
"test.arch_lm",
"ARCH-LM",
"Calm periods and volatile periods do not occur in groups.",
"volatility_clusters",
"no_volatility_clusters",
"Volatile periods occur in groups. The ranges must be wider after shocks.",
"The volatility does not occur in groups.",
),
_t(
"test.granger",
"Granger causality",
"Past values of the driver do not help to predict the result.",
"predictive",
"not_predictive",
"Past values of the driver help to predict the result.",
"Past values of the driver do not help to predict the result.",
),
_t(
"test.acf_pacf",
"Autocorrelation significance",
"No lag has a relation to the present that is more than chance.",
"autocorrelated",
"white_noise",
"Some past periods have a relation to the present (see the suggested orders).",
"No past period has a relation to the present that is more than chance.",
),
_t(
"test.seasonal_strength",
"Seasonal strength",
"There is no important pattern that repeats.",
"seasonal",
"not_seasonal",
"There is a clear pattern that repeats. Add the season to the model.",
"The test found no clear pattern that repeats.",
p_value=False,
),
_t(
"test.vif",
"Variance inflation (VIF)",
"The drivers are not too similar to each other.",
"collinear",
"not_collinear",
"Some drivers move almost together. It is difficult to separate their effects.",
"The drivers are sufficiently different to separate their effects.",
p_value=False,
),
_t(
"test.cusum",
"CUSUM stability",
"The relationship stays the same over time.",
"unstable",
"stable",
"The relationship changed at one point. Add a break, or use a shorter history.",
"The relationship is stable over time.",
),
_t(
"test.chow",
"Chow break test",
"There is no break at the selected date.",
"break",
"no_break",
"The relationship changed at the selected date.",
"The test found no change at the selected date.",
),
)
}
The statistical tests workflow test nodes run (ids match the node implementations).
Bases: ResponseModel
The test_result of one test node (and one stationarity-table row).
For panels the test runs per series: per_series holds each one and the headline
fields summarise them (reject when reject_share >= 0.5; p_value is the
median). Branch conditions read these fields, e.g. nodes.adf.reject.
Bases: ResponseModel
The test on one series of a panel (coords names it, e.g. {"geo": "DE"}).
TestDefinition(
id: str,
name: str,
null_plain: str,
reject_decision: str,
keep_decision: str,
reject_plain: str,
keep_plain: str,
p_value: bool = True,
)
What a test checks and what each outcome means, in plain words.
reject_decision/keep_decision are stable machine labels for the decision when
the null is rejected or kept (branch conditions compare nodes.<id>.decision with
them); *_plain are the sentences the UI shows.
False for tests decided on critical values only (Johansen's rank, CUSUM bounds).
(reject, decision, verdict_plain) of a p-value at significance alpha.
reject is p_value < alpha; the decision label depends on what the test's null
means (see :data:TEST_DEFINITIONS). Raises KeyError for unknown tests and
ValueError for a p-value or alpha outside [0, 1].
make_test_result(
test_id: str,
*,
statistic: float | None,
p_value: float,
alpha: float = 0.05,
var: str | None = None,
**extra: JsonValue,
) -> StatTestResult
A :class:StatTestResult with the decision and texts filled from p_value.
Governance v0.5¶
See Governance.
Model governance: policies, the check catalog, evidence, the validation state machine, approvals and the evidence pack (M3.5).
A :class:GovernancePolicy (tenant- or project-scoped, versioned) lists the checks a model
of a family must pass before it is validated, with thresholds and severities, and
whether a second person must approve. Checks run as governance workflows in the sandbox
(governance.policy nodes); their :class:CheckResult s with :class:EvidenceItem s are
stored on a :class:ModelValidation.
Validation status (:func:transition, pure and exhaustive)::
draft --submit--> checks_running
checks_running --checks_passed--> checks_passed
checks_running --checks_failed--> checks_failed
checks_running --cancel--> draft
checks_passed --validate--> validated (no approval required)
checks_passed --request_approval--> awaiting_approval (approval required)
awaiting_approval --approve--> validated
awaiting_approval --reject--> rejected
checks_failed --resubmit--> checks_running
checks_failed --reject--> rejected
rejected --resubmit--> checks_running
validated --deprecate--> deprecated
Approvals need a reviewer who is not the submitter (:func:check_approver). When a policy
says require_validated_for_promotion, promoting a model version to an endpoint needs
status validated (:func:mapi_spec.scorecard.promotion_gate, reason NOT_VALIDATED).
Every transition is audited and emitted as a webhook event.
GOVERNANCE_CHECKS: Final[dict[str, GovernanceCheckDef]] = {
c.id: c for c in _CHECKS
}
The governance check catalog by id.
TRANSITIONS: Final[
dict[
tuple[ValidationStatus, ValidationEvent],
ValidationStatus,
]
] = {
(_S.DRAFT, _EV.SUBMIT): _S.CHECKS_RUNNING,
(
_S.CHECKS_RUNNING,
_EV.CHECKS_PASSED,
): _S.CHECKS_PASSED,
(
_S.CHECKS_RUNNING,
_EV.CHECKS_FAILED,
): _S.CHECKS_FAILED,
(_S.CHECKS_RUNNING, _EV.CANCEL): _S.DRAFT,
(
_S.CHECKS_PASSED,
_EV.REQUEST_APPROVAL,
): _S.AWAITING_APPROVAL,
(_S.CHECKS_PASSED, _EV.VALIDATE): _S.VALIDATED,
(_S.CHECKS_FAILED, _EV.RESUBMIT): _S.CHECKS_RUNNING,
(_S.CHECKS_FAILED, _EV.REJECT): _S.REJECTED,
(_S.AWAITING_APPROVAL, _EV.APPROVE): _S.VALIDATED,
(_S.AWAITING_APPROVAL, _EV.REJECT): _S.REJECTED,
(_S.REJECTED, _EV.RESUBMIT): _S.CHECKS_RUNNING,
(_S.VALIDATED, _EV.DEPRECATE): _S.DEPRECATED,
}
Every allowed (status, event) -> status; anything else is refused.
Bases: ResponseModel
A versioned policy. project_id is None for tenant-wide policies.
Bases: RequestModel
POST /v1/governance/policies (project) or /v1/tenants/{id}/governance/policies.
Bases: GovernancePolicyCreate
PUT …/policies/{policy_id}: a new version (the old one stays for audit).
Bases: StrictModel
A required check in a policy: thresholds, severity and parameters.
Bases: StrictModel
Which models a policy covers; an empty list matches everything on that axis.
A family matches a model's declared family (forecast for the statsmodels
forecasters, ts_multivariate, mmm) or the namespace of its id (ts for
ts.sarimax), so families: [ts] covers every ts.* model whatever family its
plugin declares.
Bases: StrEnum
What a failing check does: blocking stops validation, warning is recorded and
shown to the approver, info is evidence only.
Bases: ResponseModel
One check of the governance catalog (GET /v1/catalog/governance-checks).
Bases: ResponseModel
The outcome of one check on one model version.
Bases: ResponseModel
One piece of evidence behind a check: a number, a small table, a plot or a stored
artifact (ref is a run artifact name or a model:// reference).
Bases: StrEnum
Bases: StrEnum
Bases: ResponseModel
Bases: RequestModel
POST /v1/validations/{validation_id}/approvals: a reviewer's decision.
Bases: RequestModel
POST /v1/models/{model_id}/versions/{version}/validations: run the checks.
Bases: RequestModel
POST /v1/validations/{validation_id}:cancel|:resubmit|:deprecate: why (a
deprecation needs a reason; it is audited).
Bases: ResponseModel
One step of a validation's history (also an audit event).
Bases: ResponseModel
A validation of one model version: status, policies, results, approvals, history.
Bases: RequestModel
POST /v1/validations/{validation_id}/evidence-pack.
Bases: ResponseModel
The audit export of a validation (format mapi.evidence-pack/v1; the HTML
rendering is a static, script-free document with the same content).
check_status(
check: GovernanceCheckDef | str,
value: float,
thresholds: CheckThresholds | None = None,
) -> CheckStatus
pass/warn/fail of a measured value (a non-finite value is error).
rollup_results(
results: Sequence[CheckResult],
required: Sequence[PolicyCheck],
) -> tuple[bool, list[str]]
(passed, failing check ids): every required blocking check has a result that
does not block (a missing result counts as a failure).
policies_for(
policies: Iterable[GovernancePolicy],
*,
project_id: str,
family: str,
model_id: str,
template: str | None = None,
) -> list[GovernancePolicy]
The active policies covering a model in project_id: tenant-wide ones and the
project's own (tenant first).
effective_checks(
policies: Sequence[GovernancePolicy],
) -> list[PolicyCheck]
The union of the policies' checks: for a check named twice, the stricter severity,
the stricter thresholds and the stricter parameters win (tenant policies are a floor
projects cannot lower; see :data:_PARAM_RULES). Tenant policies are merged first
whatever order policies comes in.
requires_validation(
policies: Iterable[GovernancePolicy],
) -> bool
Whether promotion to an endpoint needs status validated under policies.
transition(
status: ValidationStatus | str,
event: ValidationEvent | str,
*,
require_approval: bool = False,
) -> ValidationStatus
The status after event (pure). VALIDATE straight from checks_passed is
refused when the policy requires approval (use REQUEST_APPROVAL), and
REQUEST_APPROVAL when it does not. Raises VALIDATION_TRANSITION_INVALID.
after_checks(
passed: bool, *, require_approval: bool
) -> list[ValidationEvent]
The events the orchestrator applies when a check run finishes: checks_passed
then validate or request_approval; or checks_failed.
check_approver(
*,
submitter_id: str,
approver_id: str,
is_reviewer: bool,
already_approved: Iterable[str] = (),
) -> None
Refuse an approval by the submitter (SELF_APPROVAL_FORBIDDEN), by someone without
the reviewer capability (FORBIDDEN) or twice by the same person (CONFLICT).
Explorers and budget optimisation v0.5¶
Requests for the Data Explorer and the Model Explorer (M3.5).
POST /v1/datasets/{dataset_id}/versions/{version}/explore (a :class:DataExploreRequest)
and POST /v1/models/{model_id}/versions/{version}/explore (a
:class:ModelExploreRequest) start an explore run: the sandbox decodes the data (or
the bundle), computes the requested sections and writes the views of
:mod:mapi_spec.views under ui/explore/. Results are cached per (target, request
hash), so asking again is instant. :class:ExploreJob is the run's RunSpec.spec.
Bases: RequestModel
Which Data Explorer sections to compute, and how.
kpi is the result the cross-correlations are computed against (default: the
dataset's target variable); facet_by splits series by up to two dims; period
is the season length for STL and seasonal strength (default: from the frequency).
Bases: RequestModel
Which Model Explorer sections to compute. compare_with adds other model versions
(model://…) to the comparison section; backtest runs a split plan for the
backtest section (fits run as child runs).
Bases: StrictModel
RunSpec.spec of an explore run. Inputs: dataset (and schema) for a
dataset; bundle and bundle.manifest (plus compare.<k> bundles) for a model.
Precomputed UI views: small JSON documents the runner writes next to datasets and bundles and the browser reads through signed URLs (the control plane never decodes them).
Every view carries a format tag so readers can refuse versions they do not understand:
===================== ======================== ==========================================
file format content
===================== ======================== ==========================================
preview.json mapi.preview/v1 a long-format sample: dim then var columns
profile.json mapi.profile/v1 per-variable statistics + missingness grid
ui/tree_index.json mapi.tree-index/v1 structure of the bundle DataTree (no arrays)
ui/preprocess.json mapi.preprocess/v1 learned preprocessing state per pipeline step
ui/grids/mapi.response-grid/v1per-draw response curves for what-if sliders
response_curves.json (M3; :class:ResponseGridView)
ui/guided/mapi.forecast/v1a forecast score run's KPI: history, then the
forecast.json forecast fan after the split (M3)
ui/guided/mapi.counterfactual/v1a counterfactual score run's change of the
counterfactual.json KPI, in total and per period (M3)
ui/explore/mapi.model-explorer. quantised posterior rasters of the outputs
fields.jsonfields/v1 that declare a grid (:class:FieldsView)
===================== ======================== ==========================================
Views are produced inside the sandbox from untrusted data: strings are capped and readers must render them as text only.
Bases: _View
ui/explore/index.json: what an explore run produced (and why a section is
missing).
Bases: _View
Bases: _Explore
overview: shape, dims, variables and a plain-language summary.
Bases: _Explore
distributions: a histogram and summary statistics per variable.
Bases: _Explore
timeseries: every variable over time, faceted by facet_by (geo, product).
Bases: _Explore
missingness: the heatmap plus the share missing per variable.
Bases: _Explore
outliers: unusual values per variable by a robust rule.
Bases: _Explore
correlation: the matrix (matrix[i][j] for vars[i], vars[j]).
Bases: _Explore
ccf: correlation of each driver with the KPI by lag (positive lag = the driver
leads the KPI by that many periods).
Bases: _Explore
acf_pacf: autocorrelation and partial autocorrelation per series.
Bases: _Explore
stationarity: the unit-root / stationarity table (ADF + KPSS per series).
Bases: _Explore
stl: trend, season and remainder per series, with their strengths.
Bases: _Explore
vif: variance inflation per driver and the condition number.
Bases: _Explore
groups: each variable compared across the levels of by (markets, stores).
Bases: _Explore
quality: data-quality warnings in plain words (gaps, constant columns, negative
spend, duplicated periods, short history, outliers, collinear drivers).
Bases: _Explore
params: estimates with intervals (Bayesian posteriors or frequentist CIs).
Bases: _Explore
diagnostics: convergence and fit statistics.
Bases: _Explore
fit: actual vs fitted with an interval, the split line at split_at.
Bases: _Explore
residuals: residuals over time, their ACF, Ljung-Box, normality and spread tests.
Bases: _Explore
decomposition: additive components over time and their share of the total.
Bases: _Explore
response_curves: mean response per channel with a band ([channel][point];
the full per-draw grid is :class:ResponseGridView).
Bases: _Explore
irf: impulse responses of VAR/VECM models: values[i][j][h] is the response of
variables[i] to a shock in variables[j] after h periods (h = 0..horizons).
Bases: _Explore
fevd: forecast-error variance decomposition: shares[i][h][j] is the share of
variables[i]'s forecast-error variance at horizon h + 1 due to variables[j].
Bases: _Explore
fields: the outputs that declare a grid as interactive rasters (intensity maps,
images, space-time densities, vector fields), from the in-sample predictions.
Rasters are quantised to uint16 (:class:FieldRaster); one view holds at most
:data:MAX_FIELD_CELLS cells. omitted lists the grid outputs that are not shown (more
than :data:MAX_FIELDS, or not in the predictions).
Bases: _View
One output drawn as a 2-D field (see :class:mapi_spec.capabilities.GridDecl).
y_coords/x_coords label the raster rows and columns (for a downsampled raster,
the coordinate at the middle of each block). Row 0 is drawn at the top unless origin
is lower. facet names the level of each other dim of the output (one entry per
level). kind is vector when the output has a component dim.
color rgb (v0.14): the channel_dim holds red, green and blue; every frame has
an :class:RgbRaster and mean is the luminance (0.299 r + 0.587 g + 0.114 b) for the
statistics and the histogram. rgb_scale uint8 shows the values as they are;
stretch maps stats.min..max of float data to 0..255.
Bases: _View
One frame of a field (one level of its frame dim, or the only frame): the posterior
mean, sd, 5% and 95% quantiles and a few draws, each a :class:FieldRaster. For a vector
field the rasters are of the magnitude sqrt(u**2 + v**2) (per draw, then summarised)
and arrows holds the mean components.
Bases: _View
One quantised raster: height x width cells, row-major (row 0 first).
data is the base64 of little-endian uint16 codes, two bytes per cell. Code
:data:FIELD_NAN (65535) is a missing cell; a code q in 0..FIELD_LEVELS decodes
to lo + (hi - lo) * q / FIELD_LEVELS (every code is lo when lo == hi). When the
field's encoding is log10 that number is log10 of the value (values <= 0 are
missing). lo/hi are None only when every cell is missing. The step between two
codes is (hi - lo) / 65534: four to five significant digits.
Bases: _View
The mean (x, y) components of a vector field on a coarse grid (one arrow per cell).
x/y are the arrow centres in raster pixel units (column, row; fractional, 0 =
the centre of the first pixel). u is the component along increasing column index and
v along increasing row index (so with origin="lower" a positive v points up),
both [row][col] in the field's units.
Bases: _Explore
forecast: forecast fans after the fitted history (see :class:ForecastView).
Bases: _Explore
backtest: metrics per fold and horizon for the model and its benchmarks.
Bases: _Explore
comparison: several model versions side by side (table[model][metric]).
ui/explore/<section>.json: where an explore run writes a section.
decode_raster(
r: FieldRaster,
encoding: Literal["linear", "log10"] = "linear",
) -> list[float | None]
The values of a raster (row-major, None = missing), as a reader decodes them.
Budget optimisation: POST /v1/models/{model_id}/versions/{version}:optimize (plan
§8.2; M3.5), backed by mmm-framework's planning module in the sandbox.
An :class:OptimizeRequest asks how to split a budget across channels (and optionally
geos): a total budget (or a change in percent of current spend, or mode="free" to fund
every channel up to break-even), bounds per channel as multiples of current spend or in
money, portfolio groups ("digital at least 40%"), the objective (the expected KPI, a
cautious percentile such as p10, or the mean of the worst draws cvar5), a horizon
and an optional efficient frontier. The optimize run answers with an
:class:OptimizeResult: the allocation per channel [and geo] with the expected KPI and its
interval, the uplift over the current plan with the chance it is positive, the expected
regret of committing to one plan, the marginal return of every channel at the plan, and
frontier points. Everything is computed from posterior draws in the sandbox; the control
plane validates only the JSON.
Mapping onto mmm-framework (planning.budget.optimize_budget, planning.frontier
.budget_frontier, optimize_budget_by_geo): total_budget / budget_change_pct /
mode / objective / value_per_kpi map one to one; ChannelBounds.min_multiplier
/ max_multiplier are its bounds and min_spend / max_spend its
abs_bounds; groups its portfolio constraints; max_draws, n_steps and
seed its sampling controls.
Bases: RequestModel
POST …:optimize: how should the budget be split?
objective: mean (expected KPI), p<q> (the q-th percentile across posterior
draws, a cautious plan) or cvar<a> (mean of the worst a% of draws). mode:
fixed spends the budget; free funds each channel to break-even (marginal value
= 1 per unit of money, with value_per_kpi converting KPI to money) and makes the
total an output. horizon is the number of future periods the plan covers (default:
the fit window's length, as the response grid).
Bases: StrictModel
Bounds on one channel: as multiples of its current spend and/or in money (money bounds override multiples, as in mmm-framework).
Bases: StrictModel
A portfolio constraint over several channels (shares of the total and/or money).
Bases: StrictModel
Also sweep budgets from budget_min to budget_max (default 40%-200% of the
current total) in n_points steps.
Bases: ResponseModel
optimize.json: the recommended plan and its uncertainty.
Bases: ResponseModel
The plan for one channel (and geo): spend now and at the optimum, shares, the expected contribution, the marginal return at the plan and how stable the recommendation is across draws.
Bases: ResponseModel
The best plan at one total budget.
Bases: StrictModel
RunSpec.spec of an optimize run (inputs: bundle, bundle.manifest).
check_feasible(
req: OptimizeRequest,
current: dict[str, float],
*,
total: float | None = None,
) -> None
Static feasibility from the current spend per channel (OPTIMIZE_INFEASIBLE):
unknown channels, bounds whose minimums exceed the budget or whose maximums cannot
reach it, and group shares that cannot be met together.
Local development¶
Local development topology: ports, origins and environment variable names.
One place for the defaults every M1 component (API, GCS shim, runner, local executor, web app, plot sandbox) agrees on, so they can be started independently without drift. Every value can be overridden through the environment variable named next to it.
LocalDev(
env: str = DEFAULT_ENV,
database_url: str | None = None,
gcs_endpoint: str = f"http://{LOCALHOST}:{GCS_SHIM_PORT}",
api_url: str = f"http://{LOCALHOST}:{API_PORT}",
web_origin: str = f"http://{LOCALHOST}:{WEB_PORT}",
plots_origin: str = f"http://{LOCALHOST}:{PLOT_SANDBOX_PORT}",
data_dir: str = ".local",
)
Resolved local configuration. database_url=None means "start embedded pgserver".
JSON Schema export¶
Export JSON Schemas (2020-12) for every public document and contract model.
GET /schemas/modelspec.json serves :func:export_schemas output; Monaco and JSON Forms
use it, and CI snapshots it to detect drift.
Return {filename: JSON Schema}; also write the files if out_dir is given.