Spec schema reference¶
Field-by-field reference for the two documents most people write, generated from
mapi_spec.export_schemas() (JSON Schema 2020-12). Unknown fields are rejected
everywhere (extra="forbid"). Read Spec format for the prose
version with examples.
Validate as you type
Point your editor's YAML language server at the schema files below, for example
with a # yaml-language-server: $schema=modelspec.json comment.
ModelSpec¶
The declarative modelling file.
| Field | Type | Default | Description |
|---|---|---|---|
apiVersion |
mapi/v1 |
required | |
kind |
ModelSpec |
required | |
metadata |
Metadata |
required | |
data |
DataBlock |
required | |
pipeline |
list of StepCall |
– | max items 64 |
model |
ModelBlock |
required | |
outputs |
OutputsBlock |
– | |
evaluate |
EvaluateBlock |
– | |
execution |
ExecutionBlock |
– | |
calibration |
Calibration (optional) |
null |
Business priors and lift tests the model should respect (M3); needs a model whose capabilities declare them |
About¶
The plain-language card of one catalog item. Every field except the expert ones is required, so an About that validates is structurally complete; :func:about_issues adds the editorial rules (one or two sentences, no jargon in the summary, known chooser tags, related ids that exist).
| Field | Type | Default | Description |
|---|---|---|---|
title |
string |
required | A human name |
summary_plain |
string |
required | One or two sentences without jargon: what it does for you |
use_when |
list of string |
required | min items 1, max items 10 |
avoid_when |
list of string |
required | min items 1, max items 10 |
data_needs |
list of string |
required | min items 1, max items 10 |
assumptions |
list of string |
required | min items 1, max items 10 |
outputs |
list of string |
required | min items 1, max items 10 |
difficulty |
Difficulty |
required | |
typical_runtime |
TypicalRuntime |
required | |
example |
string |
required | A tiny example |
related |
list of string |
– | Ids of related catalog items (models, steps, nodes, checks, templates) (max items 16) |
references |
list of AboutReference |
– | max items 10 |
expert_notes |
string (optional) |
null |
Maths, estimator details, caveats for experts |
fits |
map of list of string |
– | Chooser tags: question id -> the answers this item suits |
AboutReference¶
A source for the expert detail (a paper, a book chapter, library documentation).
| Field | Type | Default | Description |
|---|---|---|---|
title |
string |
required | |
url |
string (optional) |
null |
AsOfSplit¶
Train on coords up to and including as_of; hold out the rest (optionally gap).
| Field | Type | Default | Description |
|---|---|---|---|
after_step |
string (optional) |
null |
Stage the split happens at (a step id or 'raw'); default: end of the stateless prefix |
allow_leakage |
boolean |
false |
Downgrade LEAKAGE_RISK to a warning (acknowledged) |
kind |
as_of |
required | |
dim |
string |
required | |
as_of |
string or integer or number or boolean |
required | |
gap |
integer |
0 |
≥ 0 |
Binding¶
Canonical name → dataset name, for dims and variables (geo: market). Names that are not bound keep their dataset name. The binding is stored in the bundle and reversed on every output by default.
| Field | Type | Default | Description |
|---|---|---|---|
dims |
map of string |
– | |
vars |
map of string |
– |
BusinessPriors¶
Beliefs in business units, per channel (keys are channel labels as in the data), and/or about the baseline. At least one belief is stated: a baseline belief alone is fine (channels without a belief keep the model's default priors).
| Field | Type | Default | Description |
|---|---|---|---|
channels |
map of ChannelBelief |
– | |
baseline_share |
ShareBelief (optional) |
null |
|
kpi_units |
string (optional) |
null |
|
spend_units |
string (optional) |
null |
Calibration¶
The calibration block of a ModelSpec: beliefs and experiment results the model should respect. prior_translation_run records which translate_priors run produced the priors in model.settings.priors (lineage only).
| Field | Type | Default | Description |
|---|---|---|---|
business_priors |
BusinessPriors (optional) |
null |
|
lift_tests |
list of LiftTest |
– | max items 500 |
prior_translation_run |
string (optional) |
null |
ChannelBelief¶
What the user believes about one channel: exactly one of roi or contribution_share, held with credibility. * roi: KPI units gained per spend unit over the fit window ([1.5, 4] = "every euro returns between 1.5 and 4 euros of revenue"); * contribution_share: the channel's share of the KPI over the fit window.
| Field | Type | Default | Description |
|---|---|---|---|
roi |
list of any (optional) | null |
|
contribution_share |
list of any (optional) | null |
|
credibility |
number |
0.8 |
Probability that the true value lies inside the range (0.8 = 'I am 80% sure') (≥ 0.5, ≤ 0.99) |
Checkpoint¶
Resumable checkpoints: every N draws (samplers) or every N epochs (v0.15: networks). Exactly one is set. The model must be checkpointable; otherwise validation warns and the run ignores it.
| Field | Type | Default | Description |
|---|---|---|---|
every_draws |
integer (optional) |
null |
|
every_epochs |
integer (optional) |
null |
v0.15: networks trained in epochs |
CollectionRead¶
How a model that reads datasets sees a collection: a seeded sample as pixels(image, y, x, channel) (ModelSpec data.collection).
| Field | Type | Default | Description |
|---|---|---|---|
max_samples |
integer |
5000 |
≥ 1, ≤ 100000 |
size |
list of any (optional) | null |
(height, width); default: the collection's size (needed when it has none) |
resize |
stretch or crop or pad |
stretch |
|
mode |
auto or gray or rgb |
auto |
|
stratify |
string (optional) |
label |
Keep the share of each level of this var or coord (none: uniform) |
seed |
integer (optional) |
null |
Default: the fit seed |
ColumnSplit¶
Hold out the rows whose value in column is one of holdout (a fixed split read from the data, v0.14). column is a coord or a 1-D data variable along dim. Values compare as text, so holdout: [1] matches a stored 1 or "1". With train set, rows in neither list are dropped; else every other row trains.
| Field | Type | Default | Description |
|---|---|---|---|
after_step |
string (optional) |
null |
Stage the split happens at (a step id or 'raw'); default: end of the stateless prefix |
allow_leakage |
boolean |
false |
Downgrade LEAKAGE_RISK to a warning (acknowledged) |
kind |
column |
required | |
dim |
string |
required | |
column |
string |
required | |
holdout |
list of string or integer or number or boolean |
– | min items 1, max items 64 |
train |
list of string or integer or number or boolean (optional) |
null |
DataBlock¶
The dataset, side tables, binding to canonical names and the split. For a tree dataset (M4), nodes picks the nodes whose variables are merged into the ONE flat dataset the pipeline sees (:meth:mapi_spec.tree.TreeSchema.flat), aligned per join. Without nodes a tree dataset's root node is read (when it has variables). nodes on a plain dataset is an error. The binding applies to the merged names. For an image collection (v0.15), collection says how a model that reads datasets sees it (a seeded sample); a model that trains on batches reads every image.
| Field | Type | Default | Description |
|---|---|---|---|
dataset |
string |
required | Dataset reference dataset://project/name@version |
side_tables |
map of SideTable |
– | |
bind |
Binding |
– | |
split |
TimeHoldoutSplit or LastPerGroupSplit or AsOfSplit or LeaveGroupOutSplit or MaskBlockSplit or ColumnSplit or StratifiedSplit (optional) |
null |
|
nodes |
map of NodeSelect (optional) |
null |
M4: tree datasets: the nodes merged into the pipeline input |
join |
exact or inner or outer |
exact |
M4: how the coords of the selected nodes align (xarray merge join) |
collection |
CollectionRead (optional) |
null |
v0.15: an image collection read by a model that reads datasets: a seeded sample as pixels(image, y, x, channel) |
Difficulty¶
How much statistical background an item needs to use well.
One of beginner, intermediate, expert.
EvaluateBlock¶
| Field | Type | Default | Description |
|---|---|---|---|
metrics |
list of string |
– | max items 32 |
ExecutionBlock¶
| Field | Type | Default | Description |
|---|---|---|---|
resource_class |
ResourceClass |
S |
|
timeout |
string |
– | Duration such as '45m', '1h30m' or '90s'. |
FitConfig¶
Platform fit settings. Plugin-specific limits come from the manifest's fit_schema.
| Field | Type | Default | Description |
|---|---|---|---|
mode |
FitMode |
evaluate_then_refit |
|
engine |
string (optional) |
null |
|
tune |
integer (optional) |
null |
|
draws |
integer (optional) |
null |
|
chains |
integer (optional) |
null |
|
target_accept |
number (optional) |
null |
|
seed |
integer (optional) |
null |
|
prior_predictive |
PriorPredictive (optional) |
null |
|
keep |
KeepConfig |
– | |
checkpoint |
Checkpoint (optional) |
null |
FitMode¶
One of evaluate_then_refit, train_only.
JsonValue¶
Type: any
KeepConfig¶
Optional groups kept in the bundle.
| Field | Type | Default | Description |
|---|---|---|---|
log_likelihood |
boolean |
false |
|
posterior_predictive |
PosteriorPredictiveKeep or boolean |
false |
|
prior |
boolean |
false |
LastPerGroupSplit¶
Hold out the last last rows along dim within each group (min_len kept).
| Field | Type | Default | Description |
|---|---|---|---|
after_step |
string (optional) |
null |
Stage the split happens at (a step id or 'raw'); default: end of the stateless prefix |
allow_leakage |
boolean |
false |
Downgrade LEAKAGE_RISK to a warning (acknowledged) |
kind |
last_per_group |
required | |
dim |
string |
required | |
within |
list of string |
required | min items 1, max items 8 |
last |
integer |
1 |
≥ 1 |
min_len |
integer |
1 |
≥ 1 |
LeaveGroupOutSplit¶
Hold out whole groups of dim (explicit groups or n random ones).
| Field | Type | Default | Description |
|---|---|---|---|
after_step |
string (optional) |
null |
Stage the split happens at (a step id or 'raw'); default: end of the stateless prefix |
allow_leakage |
boolean |
false |
Downgrade LEAKAGE_RISK to a warning (acknowledged) |
kind |
leave_group_out |
required | |
dim |
string |
required | |
groups |
list of string or integer or number or boolean (optional) |
null |
|
n |
integer (optional) |
null |
|
seed |
integer (optional) |
null |
LiftTest¶
One experiment: spend_delta extra spend on channel (optionally in one geo) from start to end (inclusive) caused kpi_delta incremental KPI. Give the measurement's uncertainty as a standard error se or a confidence interval ci at ci_level. A holdout ("turn the channel off") is a negative spend_delta with (usually) a negative kpi_delta.
| Field | Type | Default | Description |
|---|---|---|---|
id |
string (optional) |
null |
|
channel |
string |
required | |
geo |
string (optional) |
null |
|
start |
string |
required | |
end |
string |
required | |
spend_delta |
number |
required | Extra spend during the test (spend units) |
kpi_delta |
number |
required | Measured incremental KPI (KPI units) |
se |
number (optional) |
null |
Standard error of kpi_delta |
ci |
list of any (optional) | null |
Confidence interval of kpi_delta |
ci_level |
number |
0.9 |
≥ 0.5, ≤ 0.999 |
description |
string (optional) |
null |
MaskBlockSplit¶
Hold out random blocks over dims (block sizes per dim) covering fraction.
| Field | Type | Default | Description |
|---|---|---|---|
after_step |
string (optional) |
null |
Stage the split happens at (a step id or 'raw'); default: end of the stateless prefix |
allow_leakage |
boolean |
false |
Downgrade LEAKAGE_RISK to a warning (acknowledged) |
kind |
mask_block |
required | |
dims |
list of string |
required | min items 1, max items 8 |
fraction |
number |
required | > 0, < 1 |
block |
map of integer |
– | |
seed |
integer (optional) |
null |
Metadata¶
Identity and labels of a document.
| Field | Type | Default | Description |
|---|---|---|---|
name |
string |
required | |
project |
string |
required | |
labels |
map of string |
– | |
description |
string (optional) |
null |
|
about |
About (optional) |
null |
M3.5: a plain-language About card so others can choose this spec, pipeline or workflow (see mapi_spec.about) |
ModelBlock¶
The model plugin, its settings and fit configuration. settings.dim_policies is platform-reserved: a mapping dim → policy that may only tighten the plugin's policies. It is removed before plugin settings validation.
| Field | Type | Default | Description |
|---|---|---|---|
use |
string |
required | Registered id with an optional semver range |
settings |
map of JsonValue |
– | |
fit |
FitConfig |
– |
NodeSelect¶
One node a spec reads from a tree dataset (an entry of data.nodes). Its variables (all, or vars) are merged into the flat pipeline input, renamed prefix + name when prefix is given (so two nodes with the same variable names can both be read). Dims are never renamed: a dim shared by several selected nodes must be the same dim (equal :class:~mapi_spec.schema.DimSpec).
| Field | Type | Default | Description |
|---|---|---|---|
path |
string |
required | |
vars |
list of string (optional) |
null |
Default: every variable of the node |
prefix |
string (optional) |
null |
Prepended to the node's variable names in the flat dataset |
OutputsBlock¶
| Field | Type | Default | Description |
|---|---|---|---|
derived |
list of string |
– | max items 64 |
plots |
list of string |
– | max items 64 |
ui_draws |
integer |
300 |
≥ 0, ≤ 2000 |
PosteriorPredictiveKeep¶
| Field | Type | Default | Description |
|---|---|---|---|
thin |
integer |
1 |
≥ 1, ≤ 10000 |
PriorPredictive¶
| Field | Type | Default | Description |
|---|---|---|---|
draws |
integer |
required | ≥ 1, ≤ 100000 |
ResourceClass¶
Executor resource class. Ordered: S < M < L < XL < GPU. GPU is the last (highest-rank) member on purpose: a quota whose max_resource_class is below it refuses it, and :meth:next_up never offers it.
One of S, M, L, XL, GPU.
ScorePolicy¶
Which side-table version is used at score time.
One of pinned, latest, supplied.
ShareBelief¶
A share range (for example of the baseline: KPI that happens without any media).
| Field | Type | Default | Description |
|---|---|---|---|
share |
list of any | required | min items 2, max items 2 |
credibility |
number |
0.8 |
Probability that the true value lies inside the range (0.8 = 'I am 80% sure') (≥ 0.5, ≤ 0.99) |
SideTable¶
A small constant dataset (FX rates, holidays ...) used by steps. A side table that is a tree dataset (M4) reads ONE node, named by node.
| Field | Type | Default | Description |
|---|---|---|---|
ref |
string |
required | Dataset reference dataset://project/name@version |
score_policy |
ScorePolicy |
pinned |
|
node |
string (optional) |
null |
M4: the node of a tree dataset to read (needed when the table is a tree) |
StepCall¶
One pipeline step: {id, use: core.resample@^1, with: {...settings}}.
| Field | Type | Default | Description |
|---|---|---|---|
id |
string |
required | |
use |
string |
required | Registered id with an optional semver range |
with |
map of JsonValue |
– |
StratifiedSplit¶
Hold out a random fraction of the rows along dim in each level of by (the class label; v0.14). A level with one row stays in training.
| Field | Type | Default | Description |
|---|---|---|---|
after_step |
string (optional) |
null |
Stage the split happens at (a step id or 'raw'); default: end of the stateless prefix |
allow_leakage |
boolean |
false |
Downgrade LEAKAGE_RISK to a warning (acknowledged) |
kind |
stratified |
required | |
dim |
string |
required | |
by |
string |
required | |
fraction |
number |
required | > 0, < 1 |
seed |
integer (optional) |
null |
TimeHoldoutSplit¶
Hold out the end of an ordered dim: last periods, a fraction or after a coord.
| Field | Type | Default | Description |
|---|---|---|---|
after_step |
string (optional) |
null |
Stage the split happens at (a step id or 'raw'); default: end of the stateless prefix |
allow_leakage |
boolean |
false |
Downgrade LEAKAGE_RISK to a warning (acknowledged) |
kind |
time_holdout |
required | |
dim |
string |
required | |
last |
integer (optional) |
null |
|
fraction |
number (optional) |
null |
|
after |
string or integer or number or boolean (optional) |
null |
TypicalRuntime¶
Rough wall-clock time on typical data (S resource class).
One of instant, seconds, minutes, long.
ScoreRequest¶
Score a fitted model with replayed preprocessing.
| Field | Type | Default | Description |
|---|---|---|---|
apiVersion |
mapi/v1 |
required | |
kind |
ScoreRequest |
required | |
model |
string |
required | endpoint://project/alias or model://project/name@vN |
mode |
predict or forecast or counterfactual or simulate or filter or evaluate |
predict |
|
horizon |
Horizon (optional) |
null |
|
inputs_stage |
string |
raw |
|
data |
string (optional) |
null |
|
scenario |
string (optional) |
null |
|
side_tables |
map of string |
– | |
select |
map of list of string or integer or number or boolean |
– | |
interventions |
list of Intervention |
– | max items 256 |
output |
OutputSelector |
– | |
seed |
integer (optional) |
null |
|
future_covariates |
FutureCovariates (optional) |
null |
M3: how forecasts get future covariates (default: supplied) |
Aggregation¶
Reduce outputs over dims before summarising (validity depends on representation).
| Field | Type | Default | Description |
|---|---|---|---|
over |
list of string |
required | min items 1, max items 16 |
how |
sum or mean or min or max |
sum |
FutureCovariates¶
Where future values of future_required covariates come from in a forecast (M3). supplied (default) needs them in the request's data or scenario (MISSING_FUTURE_COVARIATE otherwise). The other policies let the runner extend them over the horizon, per series, inside the sandbox: carry_last repeats the last observed value, seasonal_naive repeats the value one season earlier (the time dim's yearly period), zero sets them to zero (e.g. "no spend planned").
One of supplied, carry_last, seasonal_naive, zero.
Horizon¶
| Field | Type | Default | Description |
|---|---|---|---|
dim |
string |
required | |
periods |
integer |
required | ≥ 1, ≤ 100000 |
Intervention¶
Change var on the coords selected by sel: exactly one of multiply/add/set.
| Field | Type | Default | Description |
|---|---|---|---|
var |
string |
required | |
sel |
map of string or integer or number or boolean or list of string or integer or number or boolean |
– | |
multiply |
number (optional) |
null |
|
add |
number (optional) |
null |
|
set |
number (optional) |
null |
OutputSelector¶
What a score request returns.
| Field | Type | Default | Description |
|---|---|---|---|
kind |
expected or predictive |
expected |
|
summary |
list of string (optional) |
null |
Summaries to return; default depends on representation |
draws |
integer |
0 |
≥ 0, ≤ 10000 |
thin_index |
bundle |
bundle |
|
units |
original or model |
original |
|
names |
dataset or canonical |
dataset |
|
aggregate |
Aggregation (optional) |
null |
|
vars |
list of string |
– | max items 256 |