The Table 2 Fallacy in Media Mix Models

Matthew Reda MMM

A marketing mix model regresses sales on media spend and a pile of controls — price, promotion, distribution, seasonality, competitor activity, a trend — and hands back one tidy coefficient table. The temptation is to read every row of that table as an effect: "TV returns 2.1x, and by the way price elasticity is −1.4 and December adds 12%." That second half is the Table 2 Fallacy — the epidemiologists' name for treating the adjustment coefficients in a single regression as if they carried the same causal standing as the exposure you actually designed the model to estimate (Westreich and Greenland, 2013). In an MMM the fallacy is not a curiosity. It quietly corrupts the channel ROIs everyone downstream is about to spend money against.

This is a companion to the general treatment of the Table 2 Fallacy; here we make it concrete for media measurement, where the "one model, many coefficients" habit is nearly universal and the controls are precisely the strategic, endogenous variables that break under a naive reading.

Anatomy of the MMM Table

Write the core additive MMM as a regression of a KPI \( Y_t \) on adstocked, saturated media and a set of controls:

$$ Y_t \;=\; \beta_0 \;+\; \sum_{c} \beta_c \, f_c\!\big(\mathrm{adstock}(x_{c,t})\big) \;+\; \sum_{k} \gamma_k \, z_{k,t} \;+\; \varepsilon_t, $$

where \( x_{c,t} \) is spend or impressions on channel \( c \), \( f_c \) is a saturation curve, and \( z_{k,t} \) are the controls — price, promotion depth, distribution or ACV, a seasonality basis, competitor pressure, a macro index, a trend. A Bayesian fit (Jin, Wang, Sun, Chan, and Koehler, 2017) returns a full posterior for every \( \beta_c \) and every \( \gamma_k \), and it is entirely natural to print them side by side with credible intervals. That single table is the whole trap.

Definition: The Table 2 Fallacy

Presenting the adjusted coefficients of secondary variables from one multivariable model — here the control terms \( \gamma_k \) — and interpreting each as the effect of that variable, when the model was specified to identify only the effect of the primary exposure. The secondary coefficients are typically direct effects conditional on everything else in the equation, not total effects, and they generally remain confounded even when the exposure of interest is properly adjusted (Westreich and Greenland, 2013).

The key word is conditional. Each \( \gamma_k \) is defined holding fixed every other regressor — including the media terms and the other controls. That conditional slope answers a question almost no one asked and few would want the answer to: "what is the association of price with sales, net of spend, promotion, distribution, seasonality, and trend, under whatever functional form we happened to impose?" It is not the price elasticity, not the seasonal lift, not the promotion effect. It is an adjustment term.

Even the Media Coefficients Aren't Free

Before indicting the controls, it is worth conceding that the coefficients we do want — the \( \beta_c \) that become channel ROI — are causal only under an identification argument, not by virtue of being estimated. Three assumptions are load-bearing. First, no unobserved confounding of spend and sales: nothing outside the model drives both how much a channel spent and how much the business sold. Second, correct functional form for carryover and shape, since a mis-specified adstock or saturation curve reassigns contribution across channels. Third, no reverse causality from sales to budget.

That third one is not academic. Media budgets are set by planners who react to expected demand — they flight spend into strong seasons, cut it in weak ones, and chase the very sales they are trying to explain. Spend is therefore endogenous, and an observational regression coefficient absorbs the demand signal the planner was responding to. Google's own MMM group documented the sharpest case: paid search spend is targeted at high-intent queries that would have converted anyway, so a naive MMM over-credits search unless the selection is corrected (Chen, Chan, Perry, Jin, Sun, Wang, and Koehler, 2018). And the field-experiment literature is bleak about how much signal observational advertising data even contains: across twenty-five large experiments, Lewis and Rao (2015) found individual sales so volatile relative to ad cost that ROI confidence intervals routinely spanned more than 100 percentage points — the "unfavorable economics" that make observational point estimates fragile.

The MMM back-door. Latent demand (red, unmeasured) drives both how much a planner spends and how much the business sells, so the spend–sales association X → Y is contaminated by the path X ← demand → Y. The channel coefficient is causal only if that path is blocked — which observational spend cannot guarantee.

💡 The honest reading of an MMM table

Channel coefficients are causal conditional on an identification story you are asserting, most credibly when that story is backed by experiments. Control coefficients are adjustment terms with no such story attached. Treat the first as estimates to be calibrated and the second as nuisance parameters — never as a second column of "effects."

Good Controls, Bad Controls

Why can't we just throw every plausibly relevant variable into the regression and read off whatever we like? Because a variable's role in a causal graph — not its correlation with sales — decides whether adjusting for it helps or harms. Cinelli, Forney, and Pearl (2024) organize this into a taxonomy every modeler should internalize, and it maps cleanly onto MMM controls.

Role in the DAGMMM exampleAdjust for it?
Confounder (common cause of spend and sales)Seasonality, macro demand, holiday calendarYes — omitting it biases channel effects
Mediator (on the causal path from media to sales)Brand search volume, site visits, "awareness"No, if you want the total channel effect
Collider (common effect of spend and something else)A post-spend metric driven by both media and a demand shockNever — conditioning opens a spurious path
Competing exposure (another cause of sales, not of spend)Price set independently, distribution gainsOptional — helps precision, its own coefficient stays non-causal

The crucial insight is that these roles are properties of the data-generating structure, invisible in the coefficient table itself. Two variables with identical correlations and identical fitted slopes can demand opposite treatment: adjust for the confounder, refuse to adjust for the collider. No amount of staring at t-statistics or credible intervals will tell you which is which. Only a causal diagram will — which is why the fix for the Table 2 Fallacy is, at bottom, "draw the graph first." For a longer walk through the underlying causal machinery, see the framework's causal inference primer.

The Confounder That Confounds Itself

Start with the controls we are right to include and still wrong to interpret. Seasonality is a genuine confounder: December drives both heavier spend and heavier sales, so leaving it out biases the channel betas upward. Including a seasonal basis is correct. But the seasonality coefficients themselves are not "the effect of December." They are the residual seasonal pattern net of media — and if some of that seasonal lift actually flows through advertising the brand ran because it was December, the control has already eaten part of the channel's own contribution. The seasonal coefficient and the channel coefficient are entangled by construction.

The general statement, from Westreich and Greenland, is that a confounder's own coefficient is itself typically confounded: the model was built to de-confound the exposure, and it makes no promise about de-confounding the adjustment variables. In an MMM, seasonality is confounded by promotions, promotions by price, price by competitive dynamics, and none of these were given the adjustment set that would make their coefficients causal. Each control's slope is a projection onto whatever variance the other regressors left behind. Reporting it as an effect asserts an identification you never argued for.

The Mediator Trap

The most damaging version of the fallacy in modern MMMs is not misreading a control — it is choosing the wrong controls in the first place, and the classic mistake is adjusting for a mediator. Suppose an upper-funnel channel — a brand campaign, say — works partly by lifting brand search volume, site visits, or measured awareness, which then convert to sales. That intermediate variable sits on the causal path from media to KPI:

$$ \text{Brand TV} \;\longrightarrow\; \text{brand search / awareness} \;\longrightarrow\; \text{sales}. $$

Analysts routinely toss brand search into the model as a "control" because it is a strong predictor of sales. But conditioning on a mediator blocks the indirect path and strips out exactly the mediated portion of the channel's effect — a textbook case of over-adjustment that attenuates the estimate toward zero (Cinelli, Forney, and Pearl, 2024; VanderWeele, 2015). The brand campaign looks weak, not because it is, but because the model handed its credit to the mediator sitting next to it. The channel's total effect — the number a budget decision needs — is the sum of its direct and indirect paths, and adjusting for the mediator discards the indirect one.

⚠️ Do not "control for" a downstream KPI

Any variable that media causes on its way to sales — organic/brand search, app installs, site sessions, funnel-stage awareness — is a mediator, not a control. Adjusting for it biases the upper-funnel channels toward zero and quietly reallocates their credit. If a variable is genuinely part of the mechanism, it should be modeled as a mediator, not adjusted away.

The constructive answer is to stop pretending the mediator is a nuisance and model the mechanism explicitly. A structural or nested MMM writes down the two equations — media to the mediator, mediator (plus direct media) to sales — and recovers the total effect as the sum of the paths rather than annihilating one of them. That is the difference between adjusting a mediator away and estimating through it: the first commits the Table 2 Fallacy, the second answers the mediation question the fallacy was hiding.

The Collider Trap

Colliders are subtler and rarer in MMM, but when they occur they inject bias where the modeler expected to remove it. A collider is a common effect of two variables; conditioning on it opens a non-causal association between its parents. Imagine a post-spend operational metric — inventory depletion, say, or a "marketing-qualified lead" score — that is driven both by media spend and by an unobserved demand shock. Adding it as a control conditions on a collider, opening a spurious path spend ↔ demand shock → sales and biasing every channel coefficient in a direction that is hard to sign a priori.

The general rule from the good-controls taxonomy is uncompromising: never adjust for a variable that is a descendant of the treatment and shares a hidden cause with the outcome. In practice the trap springs whenever someone reaches for a "richer" feature set of downstream engagement or operational signals to "improve fit." Fit improves; identification degrades. This is the deepest way the fallacy bites — a bad control need not merely be uninterpretable in its own row, it can poison the exposure rows you do care about.

The Price Coefficient Is Precise Nonsense

Return to the row everyone wants to read: price. Its coefficient looks like an elasticity, has a tight Bayesian credible interval, and is nearly always reported as though the model measured price sensitivity. It did not. Price in an MMM is set strategically — discounted during promotional windows, moved with competitor pricing, correlated with the very seasonality and promotion terms sitting beside it in the equation. Its coefficient absorbs whatever the model failed to specify about demand, promotion timing, and competitive response.

The Bayesian machinery makes this worse in a specific, seductive way. A narrow posterior on the price coefficient signals statistical precision — the data pin down that conditional slope tightly — but says nothing about whether the slope is the causal quantity. You can have a razor-sharp credible interval around a badly confounded number. It is precise nonsense: confident about the wrong thing.

One table, four coefficients, one identified

Every row below comes out of the same MMM with the same tidy credible interval. Only TV is anchored by a geo experiment, so its coefficient stays put. Paid search is demand-chasing — its slope inflates as latent demand drives both the queries and the sales. Price and seasonality are adjustment terms with no causal warrant; they drift too. Drag the latent-demand confounding and watch three of the four rows wander away from their true structural values (open diamonds) while their intervals stay just as narrow.

0.55
True search β+0.90
Reported search β̂—
Over-credited by—
Verdict—

Green = the one coefficient with an identification argument (experiment-anchored TV). Amber/red rows carry the same confident interval around a confounded number — precise nonsense. The demand-chaser’s ROI is over-credited exactly when demand is strongest, which is exactly when a planner leans on the model.

⚠️ A tight interval on a confounder is not evidence

Do not present the price, distribution, or competitor coefficient as an elasticity or effect, and do not let a narrow credible interval launder it into one. If you need a price elasticity, that is a different estimand requiring its own identification strategy — its own adjustment set, likely its own instruments or experiments — not a free byproduct of the media model.

One DAG, One Estimand at a Time

The discipline that dissolves the fallacy is embarrassingly simple to state and requires real thought to execute. It has four steps.

1. Draw one causal DAG. Put media, controls, sales, and the unobserved demand drivers on a diagram and commit to which arrows exist. This is where seasonality reveals itself as a confounder, brand search as a mediator, and the operational metric as a collider — distinctions the regression output cannot make. The framework's DAG-based model builder exists precisely so this graph is an explicit, versioned artifact rather than an implicit assumption buried in a variable list.

2. Choose the adjustment set for the estimand you actually want. The estimand here is incremental channel contribution — the total effect of each channel on sales. Given the DAG, the back-door criterion tells you which variables to adjust for (the confounders) and which to leave alone (the mediators and colliders). Different estimands imply different adjustment sets; there is no single "correct" set of controls independent of the question.

💡 Per-estimand reasoning is the whole game

The channel-total estimand adjusts for seasonality and macro demand and refuses to condition on brand search. A hypothetical "direct effect of TV net of search" estimand would condition on search — and would answer a different, usually less useful, question. One model cannot serve as a valid causal source for every coefficient it contains, because each would need its own adjustment set. Decide the estimand first, then the controls.

3. Report only channel effects as causal; label the rest as adjustment terms. The output table should carry channel ROIs as the causal deliverables and mark every control coefficient as a nuisance parameter that stabilizes the media estimates and nothing more. This is not pedantry — it is the difference between a media plan and a fabricated pricing recommendation.

4. Be careful what priors you place on confounder-role controls. There is a Bayesian wrinkle worth flagging. Placing a very tight prior on a confounder's coefficient — pinning it near zero, or near a "known" value — partially removes that variable's ability to soak up the confounding it was included to absorb, which re-opens the back-door path into the channel estimates. A confounder must be allowed enough posterior flexibility to actually do its de-confounding job; over-shrinking it is a quiet way to reintroduce the bias you added it to remove. The framework surfaces this as an explicit caution when a narrow coefficient prior is set on a control tagged with a confounder role.

Trust, but Calibrate

Even a well-specified DAG with the right adjustment set delivers channel coefficients that are causal only if the no-unobserved-confounding assumption holds — and for endogenous, planner-set media spend, that assumption is always partly heroic. The resolution is not to argue harder about the regression; it is to bring in a source of variation that does not depend on the model being right. Geo lift experiments and other randomized designs identify a channel's incremental effect from assignment rather than from the observational covariance, and that experimental estimate can be folded back in to calibrate the model's channel coefficients directly (see also the framework's calibration workflow).

This closes the loop the fallacy opens. The Table 2 Fallacy is really a warning that a single observational regression cannot be a universal causal instrument — most of its coefficients are adjustment machinery, and even its target coefficients rest on assumptions the data cannot check. Triangulation with experiments is how you validate the coefficients you chose to interpret, while the DAG discipline is how you decide which coefficients those are. Read the whole table and you will believe six impossible things before breakfast; read the two rows you identified and calibrated, and you have a measurement.

Takeaways

  • An MMM prints one coefficient table, but only the channel rows were designed to be causal. Reading the control rows — price, seasonality, distribution — as effects is the Table 2 Fallacy (Westreich and Greenland, 2013).
  • Control coefficients are conditional adjustment terms, not total effects, and they generally remain confounded even when the channel effects are properly adjusted.
  • Even the media coefficients are causal only under no-unobserved-confounding, correct adstock/saturation form, and no reverse causality — and planner-set spend is endogenous, so the assumption is always partly asserted.
  • The good/bad-controls taxonomy decides roles by graph position: adjust for confounders (seasonality, macro demand), never for colliders, and do not adjust for mediators like brand search if you want a channel's total effect (Cinelli, Forney, and Pearl, 2024).
  • Adjusting for a downstream mediator attenuates upper-funnel channels toward zero; model the mechanism with a structural/nested MMM instead of controlling it away.
  • A tight credible interval on a price or competitor coefficient is precise nonsense — statistical precision around a confounded quantity, not an elasticity.
  • Draw one DAG, pick the adjustment set for the incremental-contribution estimand, report only channel effects as causal, keep confounder priors loose enough to de-confound, and calibrate the channel coefficients with geo experiments.

References