Experiment Pre-Registration Memo

Fill-in template locking estimand, design, geos, power (with the power-ceiling check), stopping rule, and the decision rule — before launch.

experiment iddate draftedowner / analyst

Pre-registration discipline

This memo is written before launch. Changes after launch are amendments — dated, justified, and logged, never silently applied. Analysis is intention-to-treat by default; all deviations from this plan are logged with the readout.

Channel & hypothesis

Channel under test: channel

Hypothesis (directional, falsifiable):

hypothesis — e.g. incremental ROAS of channel X exceeds 1.0

Estimand

Precise: which counterfactual quantity, over which window.

Exactly one primary estimand, stated as a counterfactual contrast:

Incremental contribution (KPI units) over the window

Incremental ROAS (contribution per dollar) over the window

mROAS (marginal ROAS at current spend) over the window

Measurement window: start date to end date · KPI: kpi + units

Design

Randomized matched-pair geo liftTreatment geos selected for information yield; controls matched on pre-period behavior (posterior Mahalanobis distance over geo effects).

Matched-market difference-in-differencesWhen randomization is infeasible; serial correlation must enter the power calculation, with placebo checks pre-specified.

Budget-neutral randomized flighting (national data)When no geo split exists; on/off schedule randomized within the window.

Why this design for this channel and dataset:

design rationale

Treatment & control geos

Pair	Treatment geo	Control geo	Matching distance	Spillover screen \|r\| < 0.5
1	________	________	________	________
2	________	________	________	________
3	________	________	________	________
4	________	________	________	________

Matching basis: matching basis — e.g. pre-period KPI + posterior mahalanobis

Control candidates with cross-geo posterior correlation |r| > 0.5 are excluded from the donor pool (spillover screen).

Dose, duration & power

σ_y — residual sdρ — ar(1) of residualsmde — Δy to detectα — significance leveltarget power 1−βdesign effect d(t, ρ)Δspend per geo per weekduration t* (weeks)

Serial correlation inflates the variance of a T-week mean by the design effect D(T, ρ); ignoring it is the classic way to overstate the precision of difference-in-differences estimates. Spend delta is set from the saturation curve so the dose is detectable at the planned duration.

Power-ceiling check passed: lim_T→∞ Power(T) exceeds the target.ROI-posterior uncertainty sets an asymptotic power ceiling that no duration recovers. If the ceiling is below target, the design is infeasible — add geos or increase the spend delta and re-simulate; do not simply extend the test.

Pre-experiment simulation run: design recovers the MDE at target power in forward simulation from the fitted model.

Primary metric & analysis plan

Primary metric: primary metric

Estimator: estimator — matched-pair diff / did · α = alpha · Interval: interval — e.g. 90% ci

Intention-to-treat analysis by default; per-protocol only as a logged secondary.

Placebo / pre-period falsification check pre-specified.

Carryover guard: first 1–2 weeks treated as burn-in for adstocked channels.

Stopping rule

Stopping rule (pre-specified — no unplanned interim looks; any interim analysis must be written here before launch):

stopping rule — e.g. fixed horizon at t* weeks, no peeking

Decision rule

What result updates what decision — agreed before launch:

Readout	Decision
If ____________________	then ____________________
If ____________________	then ____________________
If the interval is too wide to distinguish the above	then ____________________

Calibration path

The readout (estimate + interval) feeds back into the MMM as an experiment calibration at the next refit, narrowing the tested channel's posterior. An experiment that cannot change a decision or a posterior should not run.

Sign-off

Analyst — name / date

Practice lead — name / date

Client — name / date