The Table 2 Fallacy in Media Mix Models

Matthew Reda MMM

A marketing mix model regresses sales on media spend and a pile of controls (price, promotion, distribution, seasonality, competitor activity, a trend) and hands back one tidy coefficient table. The temptation is to read every row of that table as an effect: "TV returns 2.1x, and by the way price elasticity is −1.4 and December adds 12%." That second half is the Table 2 Fallacy. Epidemiologists coined the term for the habit of treating the adjustment coefficients in a single regression as if they carried the same causal standing as the exposure you actually designed the model to estimate (Westreich and Greenland, 2013). In an MMM the fallacy is expensive. It quietly corrupts the channel ROIs everyone downstream is about to spend money against.

This is a companion to the general treatment of the Table 2 Fallacy. Here the subject is media measurement, where the "one model, many coefficients" habit is nearly universal and the controls are precisely the strategic, endogenous variables that break under a naive reading.

Anatomy of the MMM Table

Write the core additive MMM as a regression of a KPI \( Y_t \) on adstocked, saturated media and a set of controls:

$$ Y_t \;=\; \beta_0 \;+\; \sum_{c} \beta_c \, f_c\!\big(\mathrm{adstock}(x_{c,t})\big) \;+\; \sum_{k} \gamma_k \, z_{k,t} \;+\; \varepsilon_t, $$

where \( x_{c,t} \) is spend or impressions on channel \( c \), \( f_c \) is a saturation curve, and \( z_{k,t} \) are the controls: price, promotion depth, distribution or ACV, a seasonality basis, competitor pressure, a macro index, a trend. A Bayesian fit (Jin, Wang, Sun, Chan, and Koehler, 2017) returns a full posterior for every \( \beta_c \) and every \( \gamma_k \), and it is entirely natural to print them side by side with credible intervals. That single table is the whole trap.

Definition: The Table 2 Fallacy

Presenting the adjusted coefficients of secondary variables from one multivariable model (here the control terms \( \gamma_k \)) and interpreting each as the effect of that variable, when the model was specified to identify only the effect of the primary exposure. The secondary coefficients are typically direct effects conditional on everything else in the equation, not total effects, and they generally remain confounded even when the exposure of interest is properly adjusted (Westreich and Greenland, 2013).

The key word is conditional. Each \( \gamma_k \) is defined holding fixed every other regressor, the media terms and the sibling controls included. That conditional slope answers a question almost no one asked and few would want the answer to: "what is the association of price with sales, net of spend, promotion, distribution, seasonality, and trend, under whatever functional form we happened to impose?" What comes back is an adjustment term. Call it a price elasticity or a seasonal lift and you have promoted a nuisance parameter to a finding.

Even the Media Coefficients Aren't Free

Before indicting the controls, concede something about the coefficients we do want. The \( \beta_c \) that become channel ROI are causal only under an identification argument, never by virtue of having been estimated. Three assumptions are load-bearing. First, no unobserved confounding of spend and sales: nothing outside the model drives both how much a channel spent and how much the business sold. Second, correct functional form for carryover and shape, since a mis-specified adstock or saturation curve reassigns contribution across channels. Third, no reverse causality from sales to budget.

That third assumption is the one that breaks in practice. Media budgets are set by planners who react to expected demand. They flight spend into strong seasons, cut it in weak ones, and chase the very sales they are trying to explain. Spend is therefore endogenous, and an observational regression coefficient absorbs the demand signal the planner was responding to. Google's own MMM group documented the sharpest case: paid search spend is targeted at high-intent queries that would have converted anyway, so a naive MMM over-credits search unless the selection is corrected (Chen, Chan, Perry, Jin, Sun, Wang, and Koehler, 2018). The field-experiment literature is bleaker still about how much signal observational advertising data even contains. Across twenty-five large experiments, Lewis and Rao (2015) found individual sales so volatile relative to ad cost that ROI confidence intervals routinely spanned more than 100 percentage points, the "unfavorable economics" that make an observational point estimate fragile.

The MMM back-door. Latent demand (red, unmeasured) drives both how much a planner spends and how much the business sells, so the spend–sales association X → Y is contaminated by the path X ← demand → Y. The channel coefficient is causal only if that path is blocked, and observational spend cannot guarantee that.

💡 The honest reading of an MMM table

Channel coefficients are causal conditional on an identification story you are asserting, most credibly when that story is backed by experiments. Control coefficients are adjustment terms with no such story attached. Treat the first as estimates to be calibrated. Treat the second as nuisance parameters, never as a second column of "effects."

Good Controls, Bad Controls

Throwing every plausibly relevant variable into the regression and reading off whatever you like fails for a structural reason. A variable's role in the causal graph decides whether adjusting for it helps or harms, and its correlation with sales does not settle that role. Cinelli, Forney, and Pearl (2024) organize the possibilities into a taxonomy every modeler should internalize, and it maps cleanly onto MMM controls.

Role in the DAGMMM exampleAdjust for it?
Confounder (common cause of spend and sales)Seasonality, macro demand, holiday calendarYes. Omitting it biases channel effects
Mediator (on the causal path from media to sales)Brand search volume, site visits, "awareness"No, if you want the total channel effect
Collider (common effect of spend and something else)A post-spend metric driven by both media and a demand shockNever. Conditioning opens a spurious path
Competing exposure (another cause of sales, not of spend)Price set independently, distribution gainsOptional. Helps precision, and its own coefficient stays non-causal

These roles are properties of the data-generating structure, and they are invisible in the coefficient table itself. Two variables with identical correlations and identical fitted slopes can demand opposite treatment: adjust for the confounder, refuse to adjust for the collider. No amount of staring at t-statistics or credible intervals will tell you which is which. Only a causal diagram will. So the fix for the Table 2 Fallacy is, at bottom, "draw the graph first." For a longer walk through the underlying causal machinery, see the framework's causal inference primer.

The Confounder That Confounds Itself

Start with the controls we are right to include and still wrong to interpret. Seasonality is a genuine confounder: December drives both heavier spend and heavier sales, so leaving it out biases the channel betas upward. Including a seasonal basis is correct. But the seasonality coefficients themselves are not "the effect of December." They are the residual seasonal pattern net of media. If some of that seasonal lift actually flows through advertising the brand ran because it was December, the control has already eaten part of the channel's own contribution. The seasonal coefficient and the channel coefficient are entangled by construction.

The general statement, from Westreich and Greenland, is that a confounder's own coefficient is itself typically confounded: the model was built to de-confound the exposure, and it makes no promise about de-confounding the adjustment variables. In an MMM, seasonality is confounded by promotions, promotions by price, price by competitive dynamics, and none of these were given the adjustment set that would make their coefficients causal. Each control's slope is a projection onto whatever variance the other regressors left behind. Reporting it as an effect asserts an identification you never argued for.

The Mediator Trap

The most damaging version of the fallacy in modern MMMs happens before anyone reads a row. It is choosing the wrong controls in the first place, and the classic mistake is adjusting for a mediator. Suppose an upper-funnel channel (a brand campaign, say) works partly by lifting brand search volume, site visits, or measured awareness, which then convert to sales. That intermediate variable sits on the causal path from media to KPI:

$$ \text{Brand TV} \;\longrightarrow\; \text{brand search / awareness} \;\longrightarrow\; \text{sales}. $$

Analysts routinely toss brand search into the model as a "control" because it is a strong predictor of sales. But conditioning on a mediator blocks the indirect path, strips out exactly the mediated portion of the channel's effect, and lands the analysis in textbook over-adjustment that attenuates the estimate toward zero (Cinelli, Forney, and Pearl, 2024; VanderWeele, 2015). The brand campaign then looks weak. The model has handed its credit to the mediator sitting next to it. A budget decision needs the channel's total effect, the sum of its direct and indirect paths, and adjusting for the mediator discards the indirect one.

⚠️ Do not "control for" a downstream KPI

Any variable that media causes on its way to sales (organic/brand search, app installs, site sessions, funnel-stage awareness) is a mediator rather than a control. Adjusting for it biases the upper-funnel channels toward zero and quietly reallocates their credit. If a variable is genuinely part of the mechanism, model it as a mediator instead of adjusting it away.

The constructive answer is to stop pretending the mediator is a nuisance and model the mechanism explicitly. A structural or nested MMM writes down the two equations, media to the mediator and mediator (plus direct media) to sales, then recovers the total effect as the sum of the paths rather than annihilating one of them. That is the difference between adjusting a mediator away and estimating through it. The first commits the Table 2 Fallacy. The second answers the mediation question the fallacy was hiding.

The Collider Trap

Colliders are subtler and rarer in MMM, but when they occur they inject bias where the modeler expected to remove it. A collider is a common effect of two variables. Conditioning on it opens a non-causal association between its parents. Imagine a post-spend operational metric, inventory depletion or a "marketing-qualified lead" score, that is driven both by media spend and by an unobserved demand shock. Add it as a control and you have conditioned on a collider. That opens a spurious path spend ↔ demand shock → sales and biases every channel coefficient in a direction that is hard to sign a priori.

The general rule from the good-controls taxonomy is uncompromising: never adjust for a variable that is a descendant of the treatment and shares a hidden cause with the outcome. In practice the trap springs whenever someone reaches for a "richer" feature set of downstream engagement or operational signals to "improve fit." Fit improves. Identification degrades. This is the deepest way the fallacy bites, because a bad control does more than sit uninterpretable in its own row. It can poison the exposure rows you do care about.

The Price Coefficient Is Precise Nonsense

Price is the row everyone wants to read. Its coefficient looks like an elasticity, has a tight Bayesian credible interval, and is nearly always reported as though the model measured price sensitivity. It did not. Price in an MMM is set strategically: discounted during promotional windows, moved with competitor pricing, correlated with the very seasonality and promotion terms sitting beside it in the equation. Its coefficient absorbs whatever the model failed to specify about demand, promotion timing, and competitive response.

The Bayesian machinery makes this worse in a specific, seductive way. A narrow posterior on the price coefficient signals statistical precision. The data pin down that conditional slope tightly and say nothing whatever about whether the slope is the causal quantity. You can have a razor-sharp credible interval around a badly confounded number. That is precise nonsense, confidence about the wrong thing.

One table, four coefficients, one identified

Every row below comes out of the same MMM with the same tidy credible interval. Only TV is anchored by a geo experiment, so its coefficient stays put. Paid search chases demand, and its slope inflates as latent demand drives both the queries and the sales. Price and seasonality are adjustment terms with no causal warrant, and they drift too. Drag the latent-demand confounding and watch three of the four rows wander away from their true structural values (open diamonds) while their intervals stay just as narrow.

0.55
True search β+0.90
Reported search β̂…
Over-credited by…
Verdict…

Green = the one coefficient with an identification argument (experiment-anchored TV). Amber/red rows carry the same confident interval around a confounded number, which is precise nonsense. The demand-chaser’s ROI is over-credited exactly when demand is strongest, which is exactly when a planner leans on the model.

⚠️ A tight interval on a confounder is not evidence

Do not present the price, distribution, or competitor coefficient as an elasticity or effect, and do not let a narrow credible interval launder it into one. A price elasticity is a different estimand requiring its own identification strategy: its own adjustment set, and likely its own instruments or experiments. It never arrives as a free byproduct of the media model.

One DAG, One Estimand at a Time

The discipline that dissolves the fallacy is embarrassingly simple to state and takes real thought to execute. It has four steps.

1. Draw one causal DAG. Put media, controls, sales, and the unobserved demand drivers on a diagram and commit to which arrows exist. This is where seasonality reveals itself as a confounder, brand search as a mediator, and the operational metric as a collider. The regression output can make none of those distinctions. The framework's DAG-based model builder exists precisely so this graph is an explicit, versioned artifact rather than an implicit assumption buried in a variable list.

2. Choose the adjustment set for the estimand you actually want. The estimand here is incremental channel contribution, the total effect of each channel on sales. Given the DAG, the back-door criterion tells you which variables to adjust for (the confounders) and which to leave alone (the mediators and colliders). Different estimands imply different adjustment sets. There is no single "correct" set of controls independent of the question.

💡 Per-estimand reasoning is the whole game

The channel-total estimand adjusts for seasonality and macro demand and refuses to condition on brand search. A hypothetical "direct effect of TV net of search" estimand would condition on search, and would answer a different question, usually a far less useful one. One model cannot serve as a valid causal source for every coefficient it contains, because each would need its own adjustment set. Decide the estimand first, then the controls.

3. Report only channel effects as causal, and label the rest as adjustment terms. The output table should carry channel ROIs as the causal deliverables and mark every control coefficient as a nuisance parameter that stabilizes the media estimates and nothing more. That one distinction separates a media plan from a fabricated pricing recommendation.

4. Be careful what priors you place on confounder-role controls. The Bayesian wrinkle here is easy to miss. Placing a very tight prior on a confounder's coefficient (pinning it near zero, or near a "known" value) partially removes that variable's ability to soak up the confounding it was included to absorb, which re-opens the back-door path into the channel estimates. A confounder needs enough posterior flexibility to do its de-confounding job. Over-shrinking it is a quiet way to reintroduce the bias you added it to remove. The framework surfaces this as an explicit caution when a narrow coefficient prior is set on a control tagged with a confounder role.

Trust, but Calibrate

Even a well-specified DAG with the right adjustment set delivers channel coefficients that are causal only if the no-unobserved-confounding assumption holds. For endogenous, planner-set media spend, that assumption is always partly heroic. The resolution is to bring in a source of variation that does not depend on the model being right, rather than to argue harder about the regression. Geo lift experiments and other randomized designs identify a channel's incremental effect from assignment rather than from the observational covariance, and that experimental estimate can be folded back in to calibrate the model's channel coefficients directly (see also the framework's calibration workflow).

This closes the loop the fallacy opens. At bottom the Table 2 Fallacy is a warning that a single observational regression cannot be a universal causal instrument, since most of its coefficients are adjustment machinery and even its target coefficients rest on assumptions the data cannot check. The DAG discipline decides which coefficients you are entitled to interpret, and triangulation with experiments validates the ones you chose. Read the whole table and you will believe six impossible things before breakfast. Read the two rows you identified and calibrated, and you have a measurement.

Takeaways

  • An MMM prints one coefficient table, but only the channel rows were designed to be causal. Reading the control rows (price, seasonality, distribution) as effects is the Table 2 Fallacy (Westreich and Greenland, 2013).
  • Control coefficients are conditional adjustment terms, not total effects, and they generally remain confounded even when the channel effects are properly adjusted.
  • Even the media coefficients are causal only under no-unobserved-confounding, correct adstock/saturation form, and no reverse causality. Planner-set spend is endogenous, so that last assumption is always partly asserted.
  • The good/bad-controls taxonomy decides roles by graph position: adjust for confounders (seasonality, macro demand), never for colliders, and do not adjust for mediators like brand search if you want a channel's total effect (Cinelli, Forney, and Pearl, 2024).
  • Adjusting for a downstream mediator attenuates upper-funnel channels toward zero. Model the mechanism with a structural/nested MMM instead of controlling it away.
  • A tight credible interval on a price or competitor coefficient is precise nonsense. It is statistical precision around a confounded quantity, not an elasticity.
  • Draw one DAG, pick the adjustment set for the incremental-contribution estimand, report only channel effects as causal, keep confounder priors loose enough to de-confound, and calibrate the channel coefficients with geo experiments.

References