ump_spec.models.experiment

module ump-spec

Test plans and results (spec §9, EMA guide §5).

Classes

PowerAnalysis

class

class PowerAnalysis(StrictModel)

Fields

Name	Type	Default	Description
`mde`	`float \| None`	`None`
`mde_quantile`	`Literal['t', 'z']`	`'t'`
`assurance`	`float \| None`	`None`
`assurance_floor`	`Proposed`	`Proposed(value=0.8, source='§9')`
`power_at_q25`	`float \| None`	`None`
`sigma_exp`	`float \| None`	`None`
`n_geos`	`int \| None`	`None`
`duration_weeks`	`int \| None`	`None`
`carryover_floor_weeks`	`int \| None`	`None`
`design_effect`	`float \| None`	`None`
`alpha`	`float \| None`	`None`	Two-sided alpha recorded on the plan (§9; proposed default 0.10).
`power`	`float \| None`	`None`	Power recorded on the plan (§9; proposed default 0.8).
`sesoi`	`float \| None`	`None`	Smallest effect of interest (§9).
`method`	`str`	`'EMA eq. 3/15/16'`	Formula used.

Source: packages/ump-spec/src/ump_spec/models/experiment.py, line 29

Preregistration

class

class Preregistration(StrictModel)

Fields

Name	Type	Default	Description
`locked`	`bool`	`False`
`locked_at`	`datetime \| None`	`None`
`primary_metric`	`str \| None`	`None`
`analysis_method`	`str \| None`	`None`
`decision_rule`	`str \| None`	`None`
`hash`	`str \| None`	`None`	Content hash of the locked plan snapshot.

Source: packages/ump-spec/src/ump_spec/models/experiment.py, line 51

PlaceboCalibration

class

class PlaceboCalibration(StrictModel)

Fields

Name	Type	Default
`done`	`bool`	`False`
`null_windows`	`int \| None`	`None`
`fpr`	`float \| None`	`None`

Source: packages/ump-spec/src/ump_spec/models/experiment.py, line 60

TestPlan

class

class TestPlan(ArtifactDoc)

Fields

Name	Type	Default	Description
`client_id`	`str \| None`	`None`
`question`	`str`
`decision_informed`	`str \| None`	`None`
`channel`	`str`
`primary_metric`	`str`	`'revenue'`
`estimand`	`Estimand`
`design`	`Design`	`'matched_markets'`
`estimator`	`Estimator`	`'did'`
`admissibility_class`	`AdmissibilityClass`	`'A'`
`treatment_geos`	`list[str]`	`list()`
`control_geos`	`list[str]`	`list()`
`buffer_geos`	`list[str]`	`list()`
`spillover_design`	`str \| None`	`None`
`planned_spend`	`float \| None`	`None`
`arms`	`list[str]`	`list()`
`power`	`PowerAnalysis`	`PowerAnalysis()`
`placebo`	`PlaceboCalibration`	`PlaceboCalibration()`
`preregistration`	`Preregistration`	`Preregistration()`
`start`	`date \| None`	`None`
`end`	`date \| None`	`None`
`sized_from_model_id`	`str \| None`	`None`
`eig`	`float \| None`	`None`
`evoi`	`ShownNumber \| None`	`None`
`imported_from`	`dict[str, Any] \| None`	`None`
`status`	`Literal['draft', 'locked', 'running', 'readout']`	`'draft'`
`design_notes`	`list[dict[str, Any] \| str] \| None`	`None`	Notes from the design step. Written by experiments (`workflows.py`) and Newton’s actor.
`prior_source`	`dict[str, Any] \| None`	`None`	Prior used for power: source, mean, sd. Written by experiments (`workflows.py`) and Newton’s actor.
`national_auction_exposure`	`bool \| None`	`None`	True when the channel buys in a national auction (spillover risk). Written by experiments (`workflows.py`) and Newton’s actor.
`priority_snapshot_id`	`str \| None`	`None`	Priority snapshot the plan came from. Written by experiments (`workflows.py`) and Newton’s actor.
`spend_change`	`float \| None`	`None`	Planned relative spend change. Written by experiments (`workflows.py`) and Newton’s actor.
`synthetic`	`bool \| None`	`None`	True for synthetic (seeded) data. Written by experiments (`workflows.py`) and Newton’s actor.
`design_engine`	`str \| None`	`None`	What computed the design. Written by experiments (`workflows.py`) and Newton’s actor.
`mde_shown`	`dict[str, Any] \| None`	`None`	Minimum detectable effect as a ShownNumber. Written by experiments (`workflows.py`) and Newton’s actor.
`fallbacks`	`list[Any] \| None`	`None`	Fallback entries. Written by experiments (`workflows.py`) and Newton’s actor.
`notes`	`list[dict[str, Any] \| str] \| None`	`None`	Notes (code, severity, message). Written by experiments (`workflows.py`) and Newton’s actor.
`drafted_by`	`str \| None`	`None`	Agent principal that drafted the plan. Written by experiments (`workflows.py`) and Newton’s actor.
`newton_session`	`str \| None`	`None`	Newton session that drafted this plan. Written by experiments (`workflows.py`) and Newton’s actor.
`reader_findings`	`list[str] \| None`	`None`	Reader finding ids the actor drew on (§17). Written by experiments (`workflows.py`) and Newton’s actor.
`reader_findings_untrusted`	`list[str] \| None`	`None`	Reader finding ids marked untrusted text (§17). Written by experiments (`workflows.py`) and Newton’s actor.
`injection_flagged_findings`	`list[str] \| None`	`None`	Reader findings flagged as possible prompt injection (§17). Written by experiments (`workflows.py`) and Newton’s actor.

Source: packages/ump-spec/src/ump_spec/models/experiment.py, line 66

TestResult

class

class TestResult(ArtifactDoc)

Fields

Name	Type	Default	Description
`client_id`	`str \| None`	`None`
`metric_family`	`str \| None`	`None`	Legacy top-level metric family (sales_return, brand_awareness, …); the estimand’s own `metric_family` is preferred. A test result is judged on its estimand; kept for older writers (notebook 13).
`plan_id`	`str \| None`	`None`
`channel`	`str`
`admissibility_class`	`AdmissibilityClass`
`method`	`str`		DiD, TBR, SC, trimmed_match, ITT, CACE, platform_reported, …
`estimand`	`Estimand`
`applies_to`	`AppliesTo`	`AppliesTo()`
`lift`	`ShownNumber \| None`	`None`
`iroas`	`ShownNumber \| None`	`None`
`incremental_kpi`	`ShownNumber \| None`	`None`
`outcome`	`Outcome`	`'inconclusive'`
`largest_effect_ruled_out`	`float \| None`	`None`
`preregistered`	`bool`	`False`
`contaminated`	`bool`	`False`
`spillover_buffer_shift`	`bool`	`False`
`duration_weeks`	`int \| None`	`None`
`carryover_floor_weeks`	`int \| None`	`None`
`flight_start`	`date \| None`	`None`
`flight_end`	`date \| None`	`None`
`use_by_date`	`date \| None`	`None`
`half_life_source`	`Literal['measured', 'media_default']`	`'media_default'`
`validity_flag`	`Literal['active', 'superseded', 'contaminated', 'under_review']`	`'active'`
`provenance_tier`	`Literal['own-experimental', 'cross-brand-experimental', 'cross-brand-MMM', 'third-party-syndicated', 'platform-reported']`	`'own-experimental'`
`absorbed_by`	`str \| None`	`None`	Calibrated refit id that absorbed this readout.
`checks`	`dict[str, Any]`	`dict()`
`point`	`float \| None`	`None`	Convenience copy of iROAS point.
`interval`	`Interval \| None`	`None`
`model_interval`	`Interval \| None`	`None`	The recommending model’s pre-test interval (90% by default) for the same estimand, quoted before the flight. The readout coverage check (§25.5) scores the readout against it.
`model_point`	`float \| None`	`None`	The recommending model’s pre-test point.
`model_interval_source_id`	`str \| None`	`None`	Model artifact id that quoted model_interval (usually the plan’s sized_from).
`panel_id`	`str \| None`	`None`	Panel the readout was computed on. Written by experiments (`workflows.py`, `seed.py`) and Newton’s seed.
`plan_design_hash`	`str \| None`	`None`	Hash of the locked plan design. Written by experiments (`workflows.py`, `seed.py`) and Newton’s seed.
`readout_source`	`str \| None`	`None`	Where the readout came from: adapter or local. Written by experiments (`workflows.py`, `seed.py`) and Newton’s seed.
`source`	`str \| None`	`None`	Source of an imported result. Written by experiments (`workflows.py`, `seed.py`) and Newton’s seed.
`synthetic`	`bool \| None`	`None`	True for synthetic (seeded) data. Written by experiments (`workflows.py`, `seed.py`) and Newton’s seed.
`seed_tag`	`str \| None`	`None`	Seed label (Newton seed data). Written by experiments (`workflows.py`, `seed.py`) and Newton’s seed.
`notes`	`list[dict[str, Any] \| str] \| None`	`None`	Notes (code, severity, gate, message). Written by experiments (`workflows.py`, `seed.py`) and Newton’s seed.

Source: packages/ump-spec/src/ump_spec/models/experiment.py, line 151

PrioritySnapshot

class

class PrioritySnapshot(ArtifactDoc)

§11 test prioritisation snapshot: model version, joint draws used, scores, the kept portfolio (P1 cost-scaled greedy on EIG vs P2 best single test), alternatives by joint EVOI and the reason for any override. Briefs MUST NOT claim the (1 - 1/e) bound for EVOI or budget-constrained selection, so guarantee_note states what is (and is not) guaranteed. Signed off by the measurement lead.

Fields

Name	Type	Default	Description
`client_id`	`str \| None`	`None`
`model_id`	`str \| None`	`None`
`model_version`	`int \| None`	`None`
`joint_draws_used`	`bool`	`False`
`n_draws`	`int \| None`	`None`
`dependence`	`Any`	`None`
`kpi`	`str \| None`	`None`
`currency`	`str \| None`	`None`
`budget`	`Any`	`None`
`candidates`	`list[dict[str, Any]]`	`list()`	Candidate tests with eig, cost, eig_per_dollar, evoi.
`clusters`	`Any`	`None`
`p1`	`list[str]`	`list()`	Cost-scaled greedy portfolio on EIG.
`p2`	`list[str]`	`list()`	Single most informative affordable test.
`kept`	`str \| None`	`None`	Which of P1 and P2 was kept (higher EIG).
`kept_portfolio`	`dict[str, Any] \| None`	`None`
`alternatives`	`list[dict[str, Any]]`	`list()`
`override_needed`	`bool`	`False`
`override_reason`	`str \| None`	`None`
`choice`	`dict[str, Any] \| None`	`None`	Measurement lead’s logged choice: {portfolio, reason, by, at}.
`guarantee_note`	`str \| None`	`None`
`divisors`	`dict[str, float] \| None`	`None`	§11: each score term’s maximum over the candidate set, used as its divisor.
`eig_estimator`	`str \| None`	`None`	§11: the EIG estimator used.
`eig_mc_se`	`float \| None`	`None`	§11: Monte Carlo SE of the EIG estimate.
`guarantee`	`Literal['eig_constant_factor', 'none'] \| None`	`None`	§11: `none` comes with note NO_GUARANTEE. Never covers EVOI.
`fallbacks`	`list[dict[str, Any]]`	`list()`
`notes`	`list[dict[str, Any]]`	`list()`
`decision_rule`	`dict[str, Any] \| None`	`None`	Decision rule the snapshot ranks against. Written by experiments (`prioritise.py`).
`weights`	`dict[str, Any] \| None`	`None`	Ranking weights (proposed). Written by experiments (`prioritise.py`).
`cluster_threshold`	`dict[str, Any] \| None`	`None`	Clustering threshold (proposed). Written by experiments (`prioritise.py`).
`assumptions`	`list[Any] \| None`	`None`	Assumed values, each labelled. Written by experiments (`prioritise.py`).

Source: packages/ump-spec/src/ump_spec/models/experiment.py, line 223

Constants

Name	Type	Value	Description
`AdmissibilityClass`		`Literal['A', 'B', 'C', 'D', 'E']`
`ADMISSIBILITY`	`dict[str, dict[str, Any]]`	`{'A': {'name': 'own randomised holdout', 'calibrates': True, 'discount': 'non...`	G-EXP-05: only A to C calibrate incrementality. E feeds the brand family only.
`Design`		`Literal['matched_markets', 'randomised_geos', 'user_holdout', 'ghost_ads', 'p...`
`Estimator`		`Literal['trimmed_match', 'augmented_synthetic_control', 'tbr', 'did', 'itt', ...`
`Outcome`		`Literal['positive', 'negative', 'no_detectable_effect', 'inconclusive']`