Carryover and Shape Effects in Bayesian MMM

Matthew Reda MMM

Every media mix model has to answer two questions before it can say anything useful about budgets: how long does an ad's effect persist after the exposure, and how does response bend as spend grows? Jin, Wang, Sun, Chan, and Koehler's 2017 Google Research paper is the modern reference point for both — and, just as importantly, for an honest accounting of how weakly a typical MMM dataset identifies the parameters involved.

A media mix model is, at bottom, a regression of aggregate sales on media spend and controls. Two well-documented features of advertising response break the plain linear version: carryover — part of an ad's impact lands in the weeks after exposure, through delayed consumer response, inventory cycles, and word of mouth — and the shape effect — response is not linear in spend; returns diminish and the channel eventually saturates. Jin et al. (2017) model both with flexible functional forms, estimate by MCMC, and then — the part practitioners skip at their peril — run the model against simulated ground truth to show exactly where a couple of years of weekly data stops being informative.

The Model in One Equation

For weekly national data over \( t = 1, \dots, T \), with \( M \) media channels and \( C \) control variables, sales (or log-sales) are modeled as

$$ y_t \;=\; \tau \;+\; \sum_{m=1}^{M} \beta_m \,\mathrm{Hill}\!\left(x^{*}_{t,m};\, \mathcal{K}_m, \mathcal{S}_m\right) \;+\; \sum_{c=1}^{C} \gamma_c\, z_{t,c} \;+\; \epsilon_t, $$

where \( x^{*}_{t,m} \) is the carryover-transformed spend of channel \( m \), \( \mathrm{Hill}(\cdot) \) is the saturation transform, \( \beta_m \ge 0 \) is the channel coefficient, \( \tau \) is baseline sales, \( \gamma_c \) picks up controls such as price, distribution, and promotion, and \( \epsilon_t \) is white noise. Media effects are additive — no cross-channel synergy — a simplification the authors flag explicitly. Everything interesting happens inside the two transforms.

Carryover: Geometric vs Delayed Adstock

The adstock transformation replaces this week's spend with a normalized weighted average of the last \( L \) weeks of spend, so the regression sees the cumulative media pressure rather than the instantaneous flight:

Definition: Adstock (finite-window, normalized)

$$ x^{*}_{t,m} \;=\; \operatorname{adstock}\!\left(x_{t-L+1,m}, \dots, x_{t,m};\, w_m, L\right) \;=\; \frac{\sum_{l=0}^{L-1} w_m(l)\, x_{t-l,m}}{\sum_{l=0}^{L-1} w_m(l)}, $$

where \( w_m(\cdot) \ge 0 \) is a weight function and \( L \) is the maximum carryover duration, taken common across channels for simplicity. The denominator normalizes the weights so adstock is a proper weighted average and preserves the spend scale; a large \( L \) approximates an infinite window.

The paper considers two weight functions. Geometric decay puts the peak effect at the exposure week and decays by a retention rate \( \alpha_m \):

$$ w^{g}_m(l;\, \alpha_m) \;=\; \alpha_m^{\,l}, \qquad 0 \lt \alpha_m \lt 1 . $$

This is the discrete geometric distributed lag — the marketing analog of the classic Koyck lag, with \( \alpha_m \) playing the carryover coefficient's role. It suits direct-response channels where effect is strongest immediately.

Delayed adstock handles the other case: media, such as brand television, whose effect takes time to build and peaks some weeks after the exposure. The weights are

$$ w^{d}_m(l;\, \alpha_m, \theta_m) \;=\; \alpha_m^{\,(l - \theta_m)^2}, \qquad 0 \lt \alpha_m \lt 1, \quad 0 \le \theta_m \le L-1, $$

where \( \theta_m \) is the delay of the peak effect: the weight is maximized at lag \( l = \theta_m \) rather than at \( l = 0 \). The form is proportional to a normal density centered at \( \theta_m \) — the Gaussian radial kernel familiar from local regression — and setting \( \theta_m = 0 \) recovers geometric decay, so the delayed form nests the simple one. With the same retention rate, geometric adstock decays monotonically from lag zero while delayed adstock rises to a pulse around its peak lag and then falls away. In the paper's simulations and its real application the window is \( L = 13 \) weeks, long enough that the truncated weights are negligible. The adstock mathematics page works through these forms, their normalization, and their implementation in this framework in more detail.

Carryover in motion: a three-week flight

A channel runs for three weeks (shaded bars), then goes dark. The lines are the adstocked pressure the model actually regresses on. Geometric decay peaks the week spend lands and falls monotonically; delayed adstock keeps building after the flight ends, peaking around lag θ. Both responses are normalized to their own peak so you can compare shape. Drag the retention rate and the delay.

0.60
3
1-week retention60%
Geometric half-life—
Delayed peak—
Effect after flight ends—

This is exactly why the ROAS numerator (below) must be summed past the end of the flight, through week t₁ + L − 1: truncating at the last spend week discards the carryover tail and understates every channel with real adstock.

Shape: The Hill Function

Saturation is modeled with the Hill function, borrowed from pharmacology, where it originated as an empirical model of receptor binding (Hill, 1910; Gesztelyi et al., 2012):

$$ \mathrm{Hill}(x;\, \mathcal{K}_m, \mathcal{S}_m) \;=\; \frac{1}{1 + \left(x / \mathcal{K}_m\right)^{-\mathcal{S}_m}}, \qquad x \ge 0 . $$

Two parameters control the curve. \( \mathcal{K}_m \gt 0 \) is the half-saturation point: \( \mathrm{Hill}(\mathcal{K}_m) = 1/2 \) for any slope, so it marks the spend level at which the channel has delivered half its maximum effect. \( \mathcal{S}_m \gt 0 \) is the shape (slope) parameter: \( \mathcal{S}_m \gt 1 \) gives an S-shaped curve, convex near zero before saturating; \( \mathcal{S}_m \lt 1 \) gives a curve that is more sharply concave near zero and flattens faster; \( \mathcal{S}_m = 1 \) is the intermediate concave case. Scaling by the channel coefficient gives the "\( \beta \)Hill" transform, in which \( \beta_m \) is the asymptotic maximum effect:

$$ \beta_m\, \mathrm{Hill}_m(x) \;=\; \beta_m \;-\; \frac{\mathcal{K}_m^{\mathcal{S}_m}\, \beta_m}{x^{\mathcal{S}_m} + \mathcal{K}_m^{\mathcal{S}_m}} . $$

A leaner alternative is the one-parameter reach transformation of Jin, Shobowale, Koehler, and Case (2012), \( R = a - b/(G + b/a) \), which maps onto \( \beta\mathrm{Hill} \) exactly when \( \mathcal{S} = 1 \) (take \( a = \beta \), \( b = \mathcal{K}\beta \)). Estimating reach instead of the full Hill form trades away the S-shape option for fewer parameters and better identifiability — a trade that, as we'll see, the data often endorses. The framework's saturation mathematics page covers the Hill family alongside the logistic and other curvature choices; sigmoids, normal-CDF integrals, and monotone splines are viable too, and share the identifiability issue discussed below.

Shape, and why average ROAS isn’t marginal ROAS

The βHill curve (β = 1) with a movable “current spend.” The dashed secant from the origin is average return per dollar—the intuition behind ROAS. The solid tangent is what the next dollar buys—marginal ROAS. Push spend up the curve and watch the tangent flatten toward zero while the secant stays stubbornly positive: that gap is how a saturated channel keeps a healthy average ROAS long after its marginal ROAS has collapsed.

0.60
1.6
1.40
Curve shapeS-shaped
Average return (∝ ROAS)—
Marginal (∝ mROAS)—
Marginal ÷ average—

𝒮 > 1 gives the S-shape (convex then concave); 𝒮 ≤ 1 is concave throughout. Budget decisions live on the tangent; reporting tends to quote the secant — and conflating them, as the post argues, is how saturated channels keep winning budget.

Which Transform Comes First?

The combined model composes the two transforms in a fixed order: adstock first, then Hill — carryover accumulates raw spend into adstocked pressure, and saturation is applied to the accumulated quantity. Jin et al. note this ordering is appropriate when spend in any single period is small relative to the cumulative spend in the window, so that saturation is a property of sustained pressure rather than of a single week's flight. The reverse composition — saturate each week's spend, then let the saturated effect carry over — is the natural choice when weekly bursts are large enough to saturate on their own. The order is a modeling assumption, not a discovery: choose it to match how the channel is actually bought, because it changes what the shape parameters mean.

Weak Identification, Priors, and Calibration

The uncomfortable core of the paper is that the flexible \( \beta \)Hill form is essentially unidentifiable from typical MMM data. Two failure modes are demonstrated directly. First, within a finite observed spend range, very different parameter triples produce nearly the same curve: a curve with \( (\mathcal{K} = 0.5,\, \mathcal{S} = 1,\, \beta = 0.3) \) is matched almost exactly by \( (\mathcal{K} = 0.95,\, \mathcal{S} = 0.748,\, \beta = 0.393) \), the two diverging only beyond the observed range. Second, when the half-saturation point lies outside the range of spend the advertiser has ever tried, curves that agree inside the range can disagree arbitrarily outside it. The consequence is sharp: the model cannot extrapolate the response curve beyond observed spend, even when it fits beautifully within it.

Two parameter triples from the paper: (𝒦=0.5, 𝒮=1, β=0.3) and (𝒦=0.95, 𝒮=0.748, β=0.393). Inside the observed spend range (shaded) the two curves are visually indistinguishable — the data cannot tell them apart. Past the last dollar anyone has ever spent, they diverge without bound. The curve is identified within range; the parameters, and any extrapolation, are not.

💡 The curve is identified; the parameters are not

In the paper's large-sample simulations, the \( \beta \)Hill curves are recovered with very low bias and small variance while the posterior medians of \( \mathcal{K} \), \( \mathcal{S} \), and \( \beta \) individually vary widely. If your decision depends on the response curve over the historical spend range — attribution, within-range reallocation — weak parameter identification is survivable. If it depends on the parameters themselves, or on the curve outside the observed range — "what happens if we double TV?" — it is not.

Because a single MMM dataset carries little information relative to its parameter count (Chan & Perry, 2017), the Bayesian framing here is less a philosophical stance than a survival strategy: priors carry the information the data lacks. The paper's prior menu respects each parameter's support — a beta or uniform prior on the retention rate \( \alpha \in [0,1) \), uniform on the delay \( \theta \) over \( [0, L-1] \), gamma on the slope \( \mathcal{S} \), half-normal on the nonnegative coefficients \( \beta_m \), and, tellingly, a beta prior on \( \mathcal{K} \) constrained to the observed spend range, precisely because values outside it are unidentifiable.

What do priors buy? In simulations comparing a realistic two years of weekly data against an implausible sixty years, the adstock parameters \( (\alpha, \theta) \) are recovered fairly well at both sizes — but the shape parameters and the resulting curves are badly biased at two years, with the \( \beta \)Hill curves systematically underestimated. Relative bias of the curve at \( x = 1 \) ran from roughly −18% to −33% across the three simulated channels at two years, shrinking to a few percent at sixty. The mechanism is prior dominance: when the sample is small and the signal weak, the posterior looks almost like the prior, and the data cannot correct any bias the prior brings. The paper's prior-sensitivity studies (see the deep dive below) drive it home: the same prior swap that removes bias in one scenario worsens it in the mirror-image scenario. There is no universally correct prior — only one that encodes real knowledge, or one that doesn't.

That is why the practical fix is not longer history — market conditions drift faster than the sample grows, and the authors explicitly do not recommend it — but more informative priors from outside the dataset: hierarchical pooling across related brands (Wang et al., 2017) or geographies (Sun et al., 2017), and, in modern practice, calibrating the channel-effect priors against randomized lift experiments. Treating prior choice, prior-predictive checking, and experiment calibration as first-class steps rather than afterthoughts is the theme of the Bayesian workflow page, which turns the paper's cautionary findings into a checklist.

Deep divePrior sensitivity: the coefficient and half-saturation experiments

The first study varies the coefficient prior across 500 simulated datasets: half-normal(0, 1), normal(0, 1), and uniform(0, 3). The two normal-family priors give nearly identical, underestimated curves; the uniform puts more mass on large \( \beta \) and pulls the estimate up — cutting one channel's curve bias from about −18% to −1.5% — but only because the truth happened to sit above where the normal priors concentrated. In a scenario where curves were overestimated, the same wide prior would worsen the bias. The lesson is not "use flat priors"; it is that in a prior-dominated regime the posterior inherits the prior's center of mass, so the prior must be defended on substantive grounds.

The second study varies the half-saturation prior — beta(2, 2), uniform(0, 1), uniform(0, 10) — under two ground truths: \( \mathcal{K} = 0.2 \) inside the observed spend range, and \( \mathcal{K} = 2 \) outside it. The fitted curves barely move across priors, but the individual \( \mathcal{K} \) estimates swing markedly under the wide uniform. Since media effect depends on the curve rather than on \( \mathcal{K} \) in isolation, the model is fairly robust to this prior — though a tighter, knowledge-backed prior speeds sampler convergence. And when the true \( \mathcal{K} \) lies outside the data range, no prior rescues its estimate; the curve inside the range remains fine regardless. Identification is a property of the data's spend variation, and priors can compensate for it but not conjure it.

ROAS vs Marginal ROAS

Attribution metrics fall out of the fitted model counterfactually. Let \( \hat{Y}_t(\cdot\,; \Phi) \) be predicted sales as a function of channel \( m \)'s spend history, and let \( \tilde{x}_{t,m} \) be the counterfactual in which channel \( m \)'s spend is zeroed during a change period \( [t_0, t_1] \). Then ROAS — total incremental revenue per dollar of the channel's spend — is

$$ \mathrm{ROAS}_m \;=\; \frac{\displaystyle\sum_{t_0 \le t \le t_1 + L - 1} \left[\, \hat{Y}_t\!\left(x_{t-L+1,m}, \dots, x_{t,m};\, \Phi\right) - \hat{Y}_t\!\left(\tilde{x}_{t-L+1,m}, \dots, \tilde{x}_{t,m};\, \Phi\right) \right]}{\displaystyle\sum_{t_0 \le t \le t_1} x_{t,m}} . $$

Note the numerator's upper limit: the sales difference is summed through \( t_1 + L - 1 \), beyond the end of the spend change, because carryover keeps moving sales after the flight stops. Truncating at \( t_1 \) understates every channel with meaningful adstock. Marginal ROAS asks the forward-looking question instead — what does the next dollar return? With \( \bar{x}_{t,m} \) the historical spend raised by 1% during the change period,

$$ \mathrm{mROAS}_m \;=\; \frac{\displaystyle\sum_{t_0 \le t \le t_1 + L - 1} \left[\, \hat{Y}_t\!\left(\bar{x}_{t-L+1,m}, \dots;\, \Phi\right) - \hat{Y}_t\!\left(x_{t-L+1,m}, \dots;\, \Phi\right) \right]}{0.01 \displaystyle\sum_{t_0 \le t \le t_1} x_{t,m}} . $$

Under saturation these two numbers routinely disagree, and the disagreement is the whole point of modeling shape. A channel far up its Hill curve can show a healthy average ROAS — the historical dollars, including the early cheap ones, paid off — while its marginal ROAS is near zero because the next dollar lands on the flat part of the curve. Budget decisions turn on the marginal quantity; reporting decisions tend to quote the average one. Conflating them is how saturated channels keep winning budget.

⚠️ Plug in posterior samples, not posterior means

Both metrics are functions of the full parameter vector \( \Phi \). The correct Bayesian computation plugs each posterior draw \( \Phi_j \) into the formulas, yielding a posterior distribution of ROAS and mROAS to summarize with a median and credible interval. Averaging the parameters first and plugging in the means ignores the strong posterior correlations among the transform parameters and can produce simply wrong metrics.

The same machinery extends to an optimal media mix: maximize total predicted sales over the change and post-change periods subject to a budget constraint, with the flight pattern fixed so only per-channel totals vary. Optimizing each posterior draw separately yields the posterior of the optimal mix, and that posterior is the warning label: in a simulation scenario where the budget pushed the optimizer into a sparsely observed region of spend, it came out with three modes, and predicted sales varied more across posterior draws than across candidate mixes. When parameter uncertainty swamps the decision's effect, the "optimal" mix is a coin flip wearing an optimizer's badge.

Model-Selection Cautions

The right adstock and shape forms are not known a priori, so the paper fits competing specifications and compares them by BIC,

$$ \mathrm{BIC} \;=\; -2 \log \hat{\mathcal{L}} \;+\; k \log n, $$

with the maximized log-likelihood approximated by averaging over post‑burn‑in posterior draws. In the real application — 2.5 years of weekly volume sales for a shampoo advertiser, five media channels, with highly correlated controls orthogonalized first — four specifications cross {delayed, geometric} adstock with {Hill, reach} shape:

ModelAdstockShapeNegLogLikBIC
IdelayedβHill98.1298.12
IIdelayedreach98.1174.81
IIIgeometricβHill98.0674.76
IVgeometricreach98.0951.49

The likelihoods are nearly identical — the extra flexibility buys essentially no fit — so the ranking is driven entirely by the complexity penalty, and the most parsimonious specification wins. The diagnostics agree with the verdict: the delay parameter \( \theta \) is not actually estimated (its posterior is nearly the uniform prior), so the delayed form contributes parameters without information; and the flexible \( \beta \)Hill yields much wider credible intervals than reach with no better fit. Even the retention rate's posterior sits close to its prior — the data carries little information about carryover at all. Three cautions follow.

First, prefer the simplest form the data cannot reject. Flexibility that the data cannot discipline is not free: it inflates posterior uncertainty in everything downstream, from ROAS intervals to the optimal mix. Second, information criteria select the best regression, not the best causal model — BIC (or DIC, or WAIC) rewards predictive fit, and a well-fitting specification can still be causally wrong if confounders are missing or media targeting responds to demand. Third, check residuals anyway: in the shampoo application, residual autocorrelation was far below that of raw log-sales but remained significant out to roughly fifteen weeks — a signature of misspecification that functional-form shopping within this model class did not resolve; the authors point toward richer multi-stage and graphical structures as the eventual answer.

Takeaways

  • Model carryover with adstock: geometric decay when effect peaks at exposure, the delayed (Gaussian-kernel) form when the peak lags by \( \theta_m \) periods — the delayed form nests the geometric at \( \theta_m = 0 \).
  • Model shape with the Hill function: \( \mathcal{K} \) is the half-saturation point, \( \mathcal{S} \) selects concave vs S-shaped; the one-parameter reach transform is the \( \mathcal{S}=1 \) special case and often all the data can support.
  • Typical MMM data identifies the response curve only within observed spend, not the parameters — priors dominate small samples, and informative priors from pooling or experiments are the fix, not longer history.
  • Average ROAS and marginal ROAS diverge under saturation; budget decisions need the marginal number, computed per posterior draw with the post-period carryover window included.
  • Prefer parsimonious specifications, treat information criteria as regression-fit judges rather than causal arbiters, and expect the optimal-mix posterior to be too wide to trust when it extrapolates.

References