Estimands (closes gap 1)

§5NormativeDraft 2.3Latest draft

Every exchanged quantity MUST carry a seven-field estimand, and the gate engine MUST NOT combine two quantities whose estimands differ without saying how it aligned them: a registered converter where one exists, otherwise a stated assumption and a wider range. This is the change that makes “the same number” mean the same thing across Meridian, Robyn, PyMC-Marketing, in-house frameworks and vendor tables.¶

Earlier drafts defined an estimand as identity, point and interval. The review showed that identity needs splitting further, because the frameworks disagree on exactly the parts the old triple left implicit: what the comparison baseline is, and whether carryover counts.¶

The seven fields¶

Field	Values (v1)	Why it is required
`intervention`¶	Type (`remove_all`, `scale`, `add`, `marginal`), size with `size_kind` (`absolute` or `relative`), arms, population (`itt`, `exposed`, `geo_set`)	A 30% cut and a full holdout trace different parts of a concave curve
`baseline`¶	`zero_spend`, `current_plan`, `reduced_spend(x)` (written `reduced_spend` with x in `baseline_x`), `control_counterfactual`	Meridian’s ROI is against zero spend; a cut test is against current spend
`horizon`¶	`immediate`, `window(n weeks)` counted from the start of the intervention unless `anchor` says otherwise (registered anchor: `fit`, below), `total_to_decay(threshold)` (threshold required; the registered default applies when absent), `total` (alias of `total_to_decay` at the registered default threshold)	Robyn’s `calibration_scope` and PyMC-Marketing’s instantaneous lift differ here
`unit`¶	`kpi`; `per` (`spend`, `impression` or `absolute`); `currency` (ISO 4217 alpha-3; required with `per: spend`, allowed with `per: absolute`); `cost_basis` (`net` or `gross`)	A CACE per impression is not a channel ROI per dollar
`point_rule`¶	A rule name (`posterior_mean`, `posterior_median`, `map`, `estimator_point`), or `{rule, scale}` with `scale` one of `identity` or `log`	Skewed posteriors give different points
`interval`¶	Method (`hdi`, `eti`, `confidence`, `bootstrap_pct`, `placebo`) and level, plus the derived methods `envelope` (rule 2.3 and the Envelope paragraph) and `imputed` (registered widening with no alternatives). Derived methods are written only by UMP, never by an engine. `level` is the two-sided central coverage (frequentist methods) or the posterior mass (`hdi`, `eti`), in (0, 1). From draws, `eti` uses sample quantile type 7 (Hyndman and Fan, 1996) over the bundle-wide thinning index. An `hdi` whose region is not one interval MUST be sent as an ordered list of segments with note `MULTIMODAL_HDI`.	An 80% HDI and a 95% CI are not interchangeable
`schema_version`¶	Registry version of these definitions	Detects silent redefinition between refreshes

Undeclared values. On the wire, a field the source did not declare MUST be written as the string "undeclared". A reader treats an absent field, or an absent required sub-field, as "undeclared", and producers MUST write it explicitly before computing a content hash, so the absent and explicit forms hash the same. JSON null is invalid anywhere in the estimand. No estimand field, sub-field or registered optional member has a default. A unit with per: spend and no currency, or a unit without cost_basis, has that sub-field undeclared, raises G-EST-01 and is widened per rule 4. unit.currency is an ISO 4217 alpha-3 code. unit.per is one of spend, impression or absolute and carries no currency.¶

Currency. unit.currency is required when per is spend, because a return per unit of spend means nothing without the money it is counted in. It MAY be present when per is absolute, for a KPI counted in money (incremental revenue, say); there its absence means the KPI is not a money amount, and it is not undeclared. A source that reports per thousand units of spend (for example per $1k) MUST be rescaled to per one unit of the currency by its adapter, as a logged converter step, before it is written with per: spend.¶

Fit-window horizon. The anchor value fit is registered: {type: window, anchor: fit} means the window is the model’s fit window. A template written before any fit, such as an adapter’s metadata, MAY leave weeks absent. Every quantity a fit produces MUST state weeks as the length of that fit’s window, filled at fit time. weeks MUST NOT be null. A window horizon without anchor is counted from the start of the intervention, as in the table.¶

Registered optional members. Besides the seven fields, an estimand MAY carry the members below. They are registered with the seven fields in the estimand registry, so every consumer recognises them. Each is optional; an absent optional member means the producer did not state it, and it does not make the estimand undeclared unless its entry below says the member is required.¶

baseline_x (number). The parameter x of reduced_spend(x): the baseline is spend at x times the current plan, so 0.7 is spend at 70% of plan. It is required when baseline is reduced_spend; when it is absent or "undeclared" there, the baseline is undeclared. It SHOULD be absent with any other baseline, and a reader ignores it there.¶

intervention.size_kind (absolute or relative). How to read intervention.size. A relative size is a fraction of current spend (-0.30 is a 30% cut, 0.10 is 10% more); an absolute size is an amount of spend in unit.currency. It is required when size is present.¶

applies_to (object, nested inside the estimand). The scope, compared by G-EST-02 as described below. Its members are all optional: media_ids (array of section 4 media IDs), geo_ids (array of section 4 region IDs), kpi (a section 4 KPI ID), period ({start, end} calendar dates, half-open as in section 4), time_id (the time registry entry ID) and client_id (string).¶

registry_id (string). The ID of the registered estimand this one was taken from, for example E-BM-AVG-ROI. It is informative: G-EST-02 compares the canonical fields, never this ID.¶

metric_family (string). A metric family registered in the estimand registry; the current registry holds sales_return, brand_awareness, ad_recall and consideration (section 25.10). Quantities from different families are never pooled or converted into each other without a registered link.¶

field_status (object). Maps one of the seven field names to declared, undeclared or assumed, recording which fields the source declared and which UMP assumed under rule 4. A field filled under rule 4 MUST be marked assumed. It is bookkeeping, and G-EST-02 does not compare it.¶

A consumer MUST reject an estimand that carries a member it does not recognise, at any level of nesting, with ESTIMAND_UNKNOWN_FIELD, because a field it cannot read could change the quantity. Recognised members are the seven fields, their sub-fields as defined in this section, and the registered optional members above; anything else is unknown. Value objects in this section and in section 4 MUST NOT carry x- extensions (section 21).¶

Scope (media IDs, geos, period, time registry entry) sits in applies_to, nested inside the estimand object, and is compared by the same gate.¶

Canonical form and equality (G-EST-02). Before G-EST-02 compares two estimands, each MUST be reduced to canonical form:¶

(a) A contrast of the current plan against zero spend has one canonical form, intervention remove_all against baseline current_plan. scale with relative size -1.0 (factor 0) against current_plan takes the same form. The reverse orientation, baseline zero_spend with the current plan as the intervention, canonicalises to the same form with the sign of the quantity flipped, recorded as a converter step.¶
(b) horizon is one of immediate, {type: window, weeks: n} (with anchor when one is declared) or {type: total_to_decay, threshold: t}. total, or a total_to_decay with no threshold, takes the registered default threshold (0.05 of the peak adstock weight, proposed).¶
(c) undeclared never equals any value, including another undeclared.¶
(d) Interval levels compare as decimals rounded to three places.¶
(e) A value not in the registry for the declared schema_version is treated as undeclared (rule 4).¶
(f) A bare point_rule name canonicalises to {rule: <name>, scale: identity}.¶

Two estimands match when every canonical field is equal. Scopes match when the media IDs, the time registry entry, the period and the geo sets are equal. When one geo set strictly contains the other, the scopes are aligned, not equal: the representativeness check of section 10 runs, and the alignment is recorded with ESTIMAND_ALIGNED.¶

"estimand": {
  "intervention": {"type": "scale", "size": -0.30, "arms": ["down"], "population": "geo_set"},
  "baseline": "current_plan",
  "horizon": {"type": "window", "weeks": 6},
  "unit": {"kpi": "incremental_revenue", "per": "spend", "currency": "USD", "cost_basis": "net"},
  "point_rule": "posterior_median",
  "interval": {"method": "hdi", "level": 0.90},
  "schema_version": "est-registry/1.2.0"
}

How common sources map today. Adapters MUST fill these fields from the source’s documented behaviour, and leave a field undeclared when the source does not say. The rows below are the starting map; the round-trip study (section 22) confirms or corrects each.¶

Source quantity	Baseline	Horizon	Note
Meridian ROI¶	`zero_spend`	Total, including lagged effects in the model’s lag window	Meridian’s docs warn that experiments may use a non-zero counterfactual
Meridian mROI¶	`current_plan`	As ROI	Marginal: small relative increase
Robyn lift calibration¶	Test’s own	`immediate` or `total`, per `calibration_scope`	Robyn recommends `immediate` for lift tests
PyMC-Marketing lift test¶	`current_plan` at x	`immediate`	Saturation step only, no carryover; absolute-value likelihood
An in-house MMM framework lift test¶	`current_plan` or `zero_spend`	Test window	MROAS for added spend, CONTRIBUTION for holdouts of 95% or more
Geo test (DiD, TBR, SC)¶	`control_counterfactual`	Flight plus cool-down	Population is the treated geo set
User-level holdout¶	`control_counterfactual`	Flight	`itt` or `exposed` (CACE) must be stated
Vendor spend table¶	Usually `undeclared`	Usually `undeclared`	Attach-only until declared (section 14)

Return versus zero spendremove_all against a zero_spend baseline. Meridian's ROI uses this baseline. The steepest slope on this curve.
A 30% cut versus the current planscale by -30% against current_plan. A cut test is against current spend. A flatter slope here.
The next small increasemarginal against current_plan. Meridian's mROI: a small relative increase. The flattest slope here.

"ROI" is three different numbers here. On a concave curve like this one, which flattens as spend grows, the return measured from zero spend comes out highest and the next dollar lowest. All three can be right at once. They answer different questions, so the estimand names the intervention and baseline.

immediateOnly the week of the spend. Robyn recommends it for lift tests; PyMC-Marketing's lift test uses the saturation step only, no carryover.
window(n weeks)A fixed run of weeks. A geo test counts its flight plus cool-down.
total_to_decay(threshold)Everything until the carryover fades below a threshold. Meridian's ROI is also a total, counting lagged effects within the model's lag window.

The same burst of spend, counted three ways. A source that stops at the first week reports a smaller effect than one that waits for the carryover to fade. A registered converter can turn an immediate effect into a total one through the adstock kernel, run per posterior draw.

Illustrative shapes. No axis values are implied.

Figure 5.1 One channel, three different ROI questions. Pick a question to see which part of the same response curve it measures, then switch to the horizon view to see how much of the carryover each source counts. The shapes are illustrative, not data.

Combination rules¶

Two quantities whose seven fields and scope match combine directly (G-EST-02 passes).
¶
Otherwise UMP aligns them, in this order, and records which step it used:
¶
1. a registered converter (immediate to total via the adstock kernel; exposed CACE to per-dollar via cost per impression; incremental KPI to iROAS via the spend delta);
  ¶
2. when one side is a model, re-asking the model the other number’s question, for example scoring the 30% cut over six weeks instead of comparing against its zero-spend ROI;
  
  When both sides carry draws and only point_rule or interval differ, the alignment is a re-summary of the draws. It is recorded as step 2.2 and needs no widening.
  ¶
3. combining under an assumed alignment, with the interval widened to the envelope across the plausible alignments;
  ¶
4. showing both numbers side by side, uncombined, when no defensible alignment exists.
  ¶
A converter that needs model parameters runs per posterior draw, never on point estimates.
¶
A field marked undeclared takes the registered default for that source type (for a vendor MMM table, zero_spend baseline and total horizon) and is marked assumed. The result’s interval covers the plausible alternatives for that field. Strict mode turns this into a stop. Registered defaults per source type are published in the estimand registry (sources/x.y.z), each marked proposed. Where no default is registered for a source type and field, G-EST-01 and G-EST-02 fall to ladder step 4 (show side by side).

Envelope (normative). The plausible set for an assumed field is the alternatives list registered for the source type. With several assumed fields, it is their cross product, capped at the registered max_alternatives (proposed 8). Beyond the cap, the step falls to rule 2.4. For each alternative, the quantity is computed per draw where draws exist. The reported interval is [minimum over alternatives of the lower end, maximum over alternatives of the upper end] at the declared level, with method envelope. The point is the value under the registered default, labelled assumed. With no registered alternatives, the interval is widened by the registered widen_factor (proposed) and the method is imputed. An envelope or imputed interval does not claim nominal coverage: it covers the truth at the declared level only if one listed alternative holds. It MUST NOT be used as a likelihood or calibration input.
¶
Briefs, evidence cards and Newton answers print the estimand in plain words beside the number, including any assumed field (“revenue per dollar versus zero spend, assumed; the vendor did not say”).
¶

Every exchanged number carries seven fields

intervention What was changed, by how much, for whom
baseline Compared with what: zero spend, the current plan, a control
horizon How long the effect is counted
unit Per dollar, per impression, or absolute
point_rule Which single number: mean, median, MAP or the estimator's own point
interval Which range: method and level
schema_version Which version of these definitions

Scope (media, geos, period) sits beside them in applies_to and is compared by the same gate.

Fields and scope matchCombine directly. G-EST-02 passes.
1. Registered converterImmediate to total via the adstock kernel; exposed CACE to per-dollar via cost per impression; incremental KPI to iROAS via the spend delta.
2. Re-ask the modelWhen one side is a model, ask it the other number's question.
3. Assumed alignment, wider rangeCombine under an assumption; widen the interval to the envelope across the plausible alignments.
4. Side by sideShow both numbers, uncombined, when no defensible alignment exists.

Never: combined silently. UMP records which step it used.

Two numbers called "ROI" can answer different questions. Before adding, averaging or comparing them, the gate checks all seven fields. If they differ, UMP walks down this ladder and stops at the first step that works.

Figure 5.2 How UMP lines up two numbers that answer different questions. Pick a pair of numbers to see which of the seven fields differ, which step of the G-EST-02 ladder aligns them, and what happens to the range. Range widths are illustrative.

Registry. Estimand definitions live in a versioned registry owned by the measurement lead, the first object stood up in Phase 0.¶