MMM Diagnostic Checklist

EDA pre-flight, the symptom → diagnosis → action decision table, and the fix ladder — the one-pager to keep next to every fit.

EDA pre-flight

Run before the first fit. Minutes of work that flag most failures before any sample is drawn.

Outlier screen — every channel's spend max/median well under the ~8× danger line.Catches data-entry spikes and one-off bursts that crush max-based normalization and silently flatten the response curve.

Identifiability screen — spend CoV high enough (≳ 0.3) for every channel.Catches nearly always-on channels with almost no variation to learn from: expect prior-dominated, wide posteriors no matter what you do observationally. Flag them for experiments now, before any fit.

Collinearity screen — no spend pair with |corr| > 0.9 / VIF > 10 left unhandled.Catches channel splits the data cannot identify: the combined effect is identified, the split isn't.

Demand-chasing screen (the mirage table) — corr(spend, KPI) next to corr(spend, demand proxy) for every channel.Catches confounded ROAS: when spend chases demand, corr(spend, KPI) measures the bidding rule, not the ad. If spend is driven by demand, the demand proxy is a mandatory control.

Causal-role classification — every control classified: confounder (must include, don't shrink) / precision control / mediator (exclude as control) / irrelevant.Catches mediators included as controls — controlling for what a channel is bought to do destroys that channel's effect. The mediator call comes from institutional knowledge, not a correlation screen.

KPI structure read — trend, seasonality, and level breaks understood before fitting.Catches regime changes a constant-coefficient MMM can't absorb; never let media absorb a break.

Pre-specification — adstock, saturation, priors, trend/seasonality pre-specified now, before seeing results.Catches researcher degrees of freedom: structural pivots must later be chosen from symptoms, never from disliked answers.

Symptom → diagnosis → action

The decision table. Find the symptom, run the cheap check, take the pivot.

#	Symptom	Likely cause	Cheap check	Action
1	Divergences / r-hat > 1.01	Model fighting the data (misspecification, conflict-laden priors) — not merely a tuning problem	Where do divergent draws sit? Does the conflict survive `target_accept` 0.95?	Triage parameters (learning, pair plots) before re-tuning; fix the structure the conflict points at
2	One channel's HDI ≫ its estimate	Weak/flat spend (low CoV) — intercept-confounded	CoV < ~0.3?	Report the interval, or buy an experiment; no model rescues a flat series
3	Two channels' HDIs huge and their spends correlated	Collinearity — the combined effect is identified, the split isn't	Spend corr > ~0.9 / VIF > 10	Group them, or lift-test one channel to anchor the split
4	PPC fails (shape / autocorrelation)	Missing time structure: trend break, seasonality, carryover misspecification	Plot residuals over time and by season	Richer trend / seasonality / adstock — then re-check
5	PPC passes	— it certifies fit, not attribution	(none — that's the point)	Never read a green PPC as causal validation
6	Confounding robustness value (RV) low on a channel	Estimate would be nullified by a mild confounder	Does a candidate confounder that strong exist?	Find/keep the proxy control; calibrate
7	RV high but a plausible confounder is known	RV measures required strength, not existence	corr(spend, proxy), corr(proxy, KPI)	Control the proxy regardless; RV is no green light
8	Parameter prior-dominated (contraction ≈ 0, overlap ≈ 1)	The data carries no information (flat, collinear, short window)	CoV, spend correlations, window length	Experiment, or own the prior in the writeup
9	Negative contraction + big shift	Prior–data conflict — often confounding demanding an absurd effect	corr(spend, demand proxy)	Close the back-door; if it persists, calibrate
10	Estimate piled on the sign constraint (β ≈ 0 under a positive prior)	True effect negative (cannibalization) or mediated-away	Cross-product / holdout correlations; mediator inventory	Allow the sign / model the cross-effect or mediation explicitly
11	Spend max/median ≫ ~8	Data-entry spike or one-off burst → normalization trap	The 30-second max/median screen	Verify, winsorize/cap, percentile-normalize
12	KPI level break	Regime change a constant-coefficient MMM can't absorb	Plot the KPI; ask the business what happened	Dummy / piecewise trend; never let media absorb a break
13	Holiday-shaped residuals	Seasonality misspecification	Residuals by week-of-year	Holiday terms / higher Fourier order
14	A ROAS that would make the channel the best investment the company owns	Demand confounding — a mirage by construction	corr(spend, demand proxy); benchmark sanity	Treat as confounded until an experiment says otherwise
15	Naive corr(spend, KPI) ≫ any plausible ROAS	Spend chases demand — the dashboard mirage	The mirage table	The proxy control is mandatory; calibrate the channel

The fix ladder

Ordered escalation — exhaust each rung before climbing to the next.

Estimand first. Pin down what "contribution" means (counterfactual zero-out) before reading any posterior. If you can't say what number would be wrong, no diagnostic can save you.
EDA pre-flight. The mirage table, the CoV screen, the max/median outlier screen, the causal-role classification — minutes of work that can flag every failure a dataset contains before the first sample is drawn.
Structural pivots — chosen from symptoms, never from disliked answers. Demand controls help but don't identify; functional-form pivots matter only when form is the problem. Pivot at the structure the symptom points at, then re-check.
Experiments calibrate what structure can't. A lift test is surgical on the tested channel — and not contagious: untested channels stay wrong. That is why the deliverable ends with a costed test plan, not a full table of midpoints.

What green diagnostics mean

The doctrine

Convergence checks the sampler, never the attribution. Green diagnostics — clean r-hat, zero divergences, a passing posterior predictive check — validate the computation, not the causal claims. Identification comes from design: declared causal roles, pre-specified structure, and experiments.