Every causal number this framework produces — every ROI, every contribution, every budget recommendation — rests on the seven assumptions below. This page states each one formally, labels it honestly as testable or untestable, shows what it costs when it breaks (measured, not asserted), and names the framework feature that addresses it.
A marketing mix model is not a measurement device; it is an argument. The data plus a set of assumptions identify a causal effect — meaning the effect can be computed at all — and the assumptions do real work: most of them cannot be verified from the data they license conclusions about. The rest of this site demonstrates these ideas in context (the causal inference guide, the pressure-testing series, the modeling workflow). This page is the consolidated version: the contract, signed, in one place.
The failure costs below are quoted from the recorded stress-matrix scorecard
(tests/synth/results/stress_matrix.md, re-recorded 2026-06-10): synthetic worlds
with known causal ground truth, each violating exactly one assumption, fit with the
framework's defaults. A silent failure means attribution was wrong while
every check an analyst acts on — convergence and the robustness value — stayed green.
| # | Assumption | Testable? | World that breaks it | Measured cost (worst channel) |
|---|---|---|---|---|
| 1 | Conditional exchangeability (no unobserved confounding) | Untestable | unobserved_confounding | +69%, silent |
| 2 | Positivity / overlap | Partially testable | multicollinearity (nearest) | +60%, silent |
| 3 | Correct functional form | Testable vs. alternatives | adstock_misspec, saturation_misspec | 96% / +65% |
| 4 | SUTVA / no interference (geo designs) | Untestable (observational) | not simulated in the matrix | design-level risk |
| 5 | Sequential ignorability (mediation) | Untestable | stress 04 mediation worlds | 2.6× sibling over-credit |
| 6 | Exogeneity of spend (no reverse causality) | Untestable | reverse_causality | 19% (this instance) |
| 7 | Stable structure over the window | Partially testable | trend_break | -41%, silent |
Notation throughout: \(Y\) is the KPI, \(X\) media spend (a vector over channels), \(Z\) the declared confounders, \(Y(x)\) the potential outcome under spend level \(x\), and \(\varepsilon\) the outcome disturbance.
In English: once you condition on the declared confounders \(Z\), the channels' spend is as good as randomly assigned. Nothing unobserved — anticipated demand, a competitor's collapse, a buzzy product launch — drives both spend and sales. Given exchangeability, the regression adjustment the model performs equals the causal effect: \(E[Y \mid \mathrm{do}(x)] = E_Z\!\left[\,E[Y \mid X{=}x, Z]\,\right]\).
This is the load-bearing wall, and it is untestable: the data cannot tell you about variables that are not in it. No goodness-of-fit statistic, no convergence check, no holdout R² can detect a confounder you did not measure.
In the unobserved_confounding world (budgets set in anticipation of a latent
demand driver the model never sees), the worst channel — Search — was over-credited by
+69%, with 23% median error across channels and 90% intervals covering only
75% of the truths. Meanwhile r-hat was 1.01 with zero divergences: a textbook
silent failure. The companion world confounding_controlled is
the subtler trap: the confounder is in the model, but as a noisy proxy — and the
worst channel was still off by +50%, again with everything green.
CausalControlRole (src/mmm_framework/config/enums.py). Confounders
receive a wide, deliberately un-shrunk coefficient prior, precision controls a regularizing
one (model/base.py, _control_prior_sigmas), and variable selection
is structurally forbidden from shrinking a confounder — shrinking one re-opens the back
door (_build_control_betas). Variables marked mediator or collider are refused
as controls outright.
validation/validator.py, configured via CausalRefutationConfig).
Honest caveat from the scorecard: the suite's verdicts have proven config-sensitive across
recordings, so a passing refutation is not treated as an all-clear for the causal
claim — it is excluded from the silent-failure gate.
validation/sensitivity_unobserved.py), defined in the box below.
calibration/,
BayesianMMM.add_experiment_calibration) — the only mechanism on this list that
replaces the assumption rather than stress-testing it. See the
closing section.
Following Cinelli & Hazlett (2020, Making Sense of Sensitivity), the robustness value \(RV_q\) for a channel is the share of residual variance an unobserved confounder would need to explain in both the channel's spend and the KPI to reduce the channel's estimated effect by a fraction \(q\) — by default \(q = 1\), i.e. to zero. With \(f = |t|/\sqrt{\mathrm{dof}}\),
$$RV_q = \tfrac{1}{2}\left(\sqrt{f_q^4 + 4 f_q^2}\; - f_q^2\right), \qquad f_q = q f,$$
a value in \([0, 1]\) where larger means more robust. A channel with RV = 0.20 survives any
confounder explaining less than 20% of residual variance on both sides; below the fragility
threshold of 0.10 the framework flags the effect as fragile to a plausible
confounder. Two caveats the module itself insists on: the RV here is an
OLS-analogy computed from posterior mean/sd with deliberately generous degrees of
freedom, so it errs toward flagging fragility; and a high RV does not prove
the effect is causal — it only says a confounder would have to be implausibly strong to
explain it away. In the unobserved_confounding world the RVs sat at 0.15–0.22
while attribution was off by +69%: the confounder planted there was simply stronger than
the RV's comfort zone.
In English: the data must actually contain spend at (or near) the levels you ask the model about. A channel that has run at roughly constant spend every week of the window has no data support for \(\mathrm{do}(X{=}0)\) — yet "what would sales have been without this channel?" is precisely the contribution question. The model will still answer; the answer is an extrapolation priced by the priors and the assumed saturation curve, not by evidence.
This one is partially testable: whether the support exists is checkable from the data; whether the model's extrapolation beyond the support is right is not. Channels that never go dark, channels flighted in lockstep with each other, and geo panels where every region gets the same plan all fail it in different ways.
The nearest matrix world is multicollinearity — channels whose spends co-move so
tightly that the data cannot separate them: 39% median error, 60% on the worst
channel, with perfect-looking convergence. A silent failure — the
posterior happily reallocates credit among the entangled channels because no week in the data
ever distinguished them.
eda/validators.py,
check_constant_series), and heavily zero-inflated channels prompt a
"dark or unrecorded?" verification (check_zero_inflation).
validation/geo_identification.py): reports per-channel cross-geo spend
dispersion against a coefficient-of-variation threshold (default 0.15) and states its own
limit — the verdict is a necessary condition for geo-level identification, never a
sufficient one, because geo variation that follows local demand is confounded variation.
In English: the real carryover pattern is representable by the chosen adstock kernel, and the real diminishing-returns curve by the chosen saturation function. If TV's effect peaks two weeks after airing but the model only allows decay-from-day-one, or if a channel has a threshold ("S-shaped") response but the model assumes concavity everywhere, the misfit lands in the causal estimates.
Functional form is testable only against the alternatives you bother to fit. You can compare a geometric kernel to a delayed one; you cannot certify that no better shape exists outside your menu.
The two form worlds are the matrix's most violent. adstock_misspec (true effect
peaks late; model assumes immediate decay): 80% median error, 96% on the worst
channel, total media contribution off by −68% — though here the diagnostics did
bite: posterior predictive checks failed and two channels' robustness values fell below the
fragility threshold, so the matrix scores it flagged, not silent.
saturation_misspec (true response has a threshold; model assumes concavity):
43% median error, +65% on the worst channel, 90% intervals covering only 25%
of truths — with PPC and refutations both green. A silent failure.
transforms/adstock.py,
AdstockKind) — the delayed and Weibull families exist precisely because
geometric decay cannot represent a late peak.
SaturationConfig
(config/transforms.py): logistic, Hill, Michaelis–Menten, tanh, or none,
with optional data-anchored bounds on the Hill half-saturation point
(compute_kappa_bounds_from_data) to keep the curve's elbow inside the region
the data covers.
validation/config.py, SensitivityConfig with
include_specification_tests): refit under alternative forms and report how
much the conclusions move. Robust findings persist across reasonable shapes; fragile ones
are the tell.
In English: when a geo experiment holds out media in Denver, Denver's sales respond only to Denver's treatment — no national TV halo bleeding across DMA lines, no retargeting pools chasing users across markets, no treated-market buzz lifting the control. If spillover flows from treated to control geos, the contrast is diluted and the measured lift understates the truth (or worse, if controls are contaminated asymmetrically, biases it in either direction).
In observational data this is untestable — you never observe the same geo with and without its neighbors' treatment. The stress matrix does not simulate it; the risk is a property of experiments, and the defense is design-level, not model-level.
planning/design.py, geo_lift_design): geos are residualized on
the model's own baseline structure, paired by residual co-movement via minimum-weight
perfect matching, randomized within pairs, and powered against a placebo simulation run on
actual history — tight pairs and design-based SEs reduce the room interference has to hide in.
planning/design.py,
flighting_design), trading geographic contrast for temporal contrast.
In English: when a mediated model decomposes TV's effect into TV → awareness → sales versus TV → sales directly, it needs no-unobserved-confounding twice: once on the media→mediator path and again on the mediator→outcome path. Anything unobserved that drives both awareness and sales — PR coverage, word of mouth, distribution wins — corrupts the split.
Untestable, and strictly harder than assumption 1: randomizing media spend fixes the first condition but not the second, because you cannot randomize awareness. The direct/indirect split is identified only by this assumption plus the priors.
The extension-traps pressure test measures both
edges of the trade. Ignoring mediation is structurally wrong: the base model credits a fully
mediated TV channel with ROAS 0.22 against a true 2.14 — the contribution flows to whoever
else touches the mediator. But NestedMMM is no free lunch: it recovers the lost
channel while over-crediting its sibling channel 2.6× — a
prior-determined split the data cannot identify — and a deliberately wrong mediator
fit cleanly, reporting a confident mediated share with green sampling diagnostics throughout.
NestedMMM (mmm_extensions/models/nested.py)
makes the mediation structure explicit and reports direct vs. mediated decompositions
(MediationEffects) — the assumption is at least declared, not smuggled.
calibration/): a lift test anchors the total causal effect even when the
mediation split remains a modeling assumption.
In English: this week's spend was not chosen by looking at this week's sales. When budgets chase performance — spend up because sales are up, cut because sales dipped — causality runs backwards through the very regression that is supposed to measure the forward effect. This is assumption 1's most common concrete mechanism in marketing, called out separately because it is the one analysts can actually interrogate institutionally: ask how the budget was set.
Untestable from the time series alone — simultaneity leaves no signature a fit statistic can isolate. It is, however, discoverable: budget-setting processes are documented human decisions.
In the matrix's reverse_causality world (spend reacting to recent sales), the
fitted model happened to recover within 19% on the worst channel with full
coverage — though the placebo refutation flagged it. Do not read that as safety: the
anticipatory variant is exactly the unobserved_confounding world
(+69%, silent), because budgeting on forecasted demand is reverse
causality with a lag. And it happens in the wild: the framework's real-data pressure test on
the Lydia Pinkham series (nbs/validation/pinkham_real_data.ipynb, 1907–1960 — the
classic dataset where advertising stood in for a sales force) confronts budget-follows-sales
feedback in the historical record itself: advertising tracks revenue across the boom,
Depression, and postwar decline, and Hanssens (1980) demonstrated the sales→advertising
feedback formally — which is exactly why that notebook's forecast grade is reported as a
measured failure rather than explained away.
CausalControlRole.CONFOUNDER converts anticipatory budgeting
back into conditional exchangeability.
validation/sensitivity_unobserved.py)
quantifies how strong the un-modeled feedback would have to be to zero out each channel.
planning/design.py — randomized geo assignment, randomized flighting)
cannot have been set by sales.
In English: the baseline the model subtracts before crediting media is assumed smooth and stationary. A distribution win, a price restructure, a pandemic, an evolving seasonal pattern — any structural shift the trend term cannot bend around gets absorbed by whatever can bend, and the most flexible terms in the model are the media curves.
Partially testable: out-of-time backtests and split-window refits can detect instability after the fact, but they cannot certify that the next window resembles the last, and in-sample checks barely react at all.
The trend_break world (a level shift mid-window under a smooth global trend):
27% median error, −41% on the worst channel, with 100% interval
coverage and clean convergence — a silent failure, the misattributed break
hiding comfortably inside wide intervals. Its sibling seasonality_misspec
(evolving seasonality vs. static low-order Fourier) is worse on the extremes: 37% median,
+104% on the worst channel, also silent.
model/trend_config.py):
Prophet-style piecewise-linear trend with Laplace-shrunk changepoints
(default 10 candidates) and cubic B-spline trends, so a real break has somewhere to go
other than the media coefficients.
validation/backtest.py): on the
trend-break world the out-of-time protocol degrades from 3.0% to 11.4% MAPE with 58%
coverage — the backtest sees the misspecification that the green in-sample checks miss.
It is not a guarantee. Listing assumptions does not weaken them, satisfy them, or substitute for them; a signed contract is still a contract whose terms can be breached. Four of the seven clauses above are flatly untestable from observational data, and the scorecard shows what breach costs: worst-channel errors of 50–100% delivered with immaculate diagnostics.
Green diagnostics validate the computation, not the causal claim. R-hat, divergences, and posterior predictive checks tell you the sampler faithfully explored the posterior of the model you wrote down — they are silent on whether the model you wrote down identifies the effect you care about. Every silent failure in the stress matrix sampled beautifully.
There is exactly one mechanism in this framework that reduces the contract rather than
monitoring it: experiment calibration. A randomized geo holdout or flighting
test manufactures its own exchangeability — assignment by coin flip cannot be confounded,
anticipated, or reverse-caused — and folding its result into the model
(add_experiment_calibration, prior or likelihood route) replaces an assumption
with a measurement for the channels and windows it covers. Everything else on this page makes
the assumptions explicit, stress-tested, and priced. Only experiments make them smaller.
That loop — which assumption to buy down next, how to design the test, and how the evidence feeds back — is the subject of the closed-loop measurement & calibration guide.
See the assumptions broken on purpose, or learn the workflow that holds them up.