Workshop 03 — Your First MMM

You know what a posterior is (workshop 00), where priors come from (workshop 01), and how a sampler finds the posterior — and how to check that it did (workshop 02). Time to point all that machinery at the question your stakeholders actually care about: which marketing channels drive sales, and by how much? That question has a name — Marketing Mix Modeling (MMM) — and on this page you fit your first one, end to end, on a synthetic world whose true answer is known to the decimal. At the end, we open the answer key and grade ourselves.

1 · The vocabulary in one sitting

Every MMM tells the same generative story about a week of sales, and the field's vocabulary is just names for the boxes in that story. You spend money on a channel (one marketing lever with its own budget line — TV, Search, Social, Display). Part of the effect lands this week and part carries over into the next few weeks — that memory is adstock. Doubling spend does not double the effect — the response bends and flattens, and that bending is saturation (diminishing returns). The bent, carried-over signal gets multiplied by an effect size \( \beta \) — how strongly the channel moves the KPI (the business outcome being modeled; here, weekly sales). Sum across channels, add the baseline — everything that would have happened with zero marketing: intercept, trend, seasonality, controls like price — and you have the week's expected KPI. Every box hides parameters the model does not know, and every one of them gets a prior and a posterior.

Computed live in your browser — the same math as the notebook. Left: one pulse of spend in week 3 echoes forward under geometric adstock with weights \( \propto \alpha^k \) over 8 weeks, normalized to sum to 1 — adstock redistributes spend in time without inventing any. Right: the saturation curve \( 1 - e^{-\lambda x} \) on max-normalized spend — bigger \( \lambda \) flattens sooner, so heavy weeks waste money; the dashed line is what returns would look like if they never bent.

Two more terms complete the kit. A channel's contribution is the KPI it caused — how much lower sales would have been had that channel spent zero. That is a counterfactual claim, a causal question rather than a correlation. And ROAS (return on ad spend) is contribution divided by spend — "KPI units bought per unit of money." The baseline is what makes the causal question hard: the model must split observed sales between "marketing did it" and "it would have happened anyway," and nothing in the data labels which is which.

2 · Meet the panel data

dgp.build("clean") manufactures three years of weekly brand history: 156 weeks (2021-01-04 → 2023-12-25), four media channels, a price control, and a sales KPI with mean 438 and standard deviation 53. It is the positive control of this repo's stress-testing suite — a world generated from exactly the assumptions the model makes, so the model has a fair shot at full marks. The stress series is about worlds that break those assumptions; the clean world is the same one it grades against. The scenario object carries an answer key, sc.true_contribution — each channel's true causal contribution, computed by re-running the world's own generating equations with that channel's spend set to zero. We keep it sealed until section 7.

The MFF format: how data enters the framework

Real engagements deliver data as an MFF (Master Flat File) — a long (tidy) table with one row per variable × period observation, rather than one column per variable. Long format looks wasteful for a tiny world, but it scales gracefully when variables arrive at different granularities (national TV, city-level Search, store-level sales): each row simply states which variable, when, where, and what value. The first week of our world, in both shapes:

Same numbers, two shapes — the notebook's assert checks the reshape is exact: 6 variables × 156 weeks = 936 MFF rows, no row lost or invented. sc.panel() wraps the data into the PanelDataset structure BayesianMMM consumes — the same thing MFFLoader produces from a real MFF file.

Look before you model

Never fit before you look. Things a working analyst eyeballs in the chart below: seasonality (the KPI breathes on an annual rhythm — repeating humps about 52 weeks apart), trend (a gentle upward drift across the three years), flighting (media bought in on-off bursts rather than a flat drip — each channel pulses on its own cycle), and dark weeks (spend repeatedly drops to near zero; those weeks anchor the baseline by showing what sales look like with almost no help). The pulse misalignment is a gift: it is what lets a model tell the channels apart. When channels flight together, attribution degenerates — stress 03 is about exactly that.

Illustrative recreation (seeded) — the same generative equations as dgp.build("clean") (pulsed flighting cycles of 9/7/11/13 weeks, trend + annual seasonality + price baseline, geometric adstock, \(1-e^{-\lambda x}\) saturation, Gaussian noise), re-drawn in your browser with a different random seed. Hover to read values; note the misaligned pulses and the near-dark weeks in every channel.

Measured — from the baked notebook run. Left: total spend by channel (TV 9,193 · Search 5,773 · Social 4,703 · Display 3,669 — budget shares 39% / 25% / 20% / 16%). Right: the same-week spend–KPI correlation per channel (TV 0.33, Search 0.38, Social 0.42, Display 0.15) — weak everywhere, even though every channel genuinely works. The maximum cross-channel spend correlation is just 0.10: distinct flighting cycles keep the channels tellable-apart.

Sit with that right-hand panel, because it carries the motivating lesson of the whole series. Every channel in this world genuinely drives sales — we built it that way — yet same-week correlation is nearly a shrug. Three things hide the signal: a TV pulse's effect arrives spread over several weeks (adstock), big-spend weeks are flattened by saturation, and the baseline (season, trend, price) moves sales around regardless of media. A naive "correlate spend with sales" analysis would find little. The MMM's job is to unbend, un-lag, and un-confound — and to report its remaining uncertainty honestly.

3 · Adstock and saturation, hands on

Two parameters make media modeling special, and the model will learn both per channel:

• Adstock retention \( \alpha \) (0 → 1): the fraction of this week's effect that survives into next week. \( \alpha = 0 \) means no memory; \( \alpha = 0.8 \) means a long echo. Geometric adstock spreads each week's spend over the next 8 weeks with weights \( \propto \alpha^0, \alpha^1, \alpha^2, \dots \), normalized to sum to 1.
• Saturation strength \( \lambda \): how quickly returns diminish, via \( 1 - e^{-\lambda x} \) on max-normalized spend. Small \( \lambda \) ≈ a straight line (no saturation in the observed range); large \( \lambda \) = the curve flattens early, so heavy weeks waste money.

The explorer below is the page's hands-on centerpiece — the same ipywidgets exhibit the notebook runs live. Drag the sliders and watch two things at once: left, the transformed series the model would actually multiply by \( \beta \); right, the response curve with the real spend-weeks scattered along it, so you can see where on the curve this channel actually lives. Try \( \alpha \to 0.9 \) on Search (a fast-twitch channel suddenly behaves like brand TV), \( \lambda \to 4 \) on TV (most weeks pile onto the flat part — extra spend buys nothing), or \( \lambda \to 0.3 \) (saturation effectively off).

Computed live in your browser — the same transform math as the notebook (geometric adstock, then \( 1-e^{-\lambda x} \)), applied to the seeded illustrative spend series above over a 60-week window. The model does not know \( \alpha \) and \( \lambda \): it puts priors on both, per channel, and learns them from the data — these sliders are your hands on knobs that NUTS (workshop 02) will soon turn by posterior inference. The world generated TV with α = 0.60, λ = 1.6.

4 · Writing the model down: ModelConfig, choice by choice

A model configuration is not bureaucracy — it is section 1's generative story written as code, plus the priors from workshop 01. The notebook's entire setup:

from mmm_framework.config import InferenceMethod, ModelConfig
from mmm_framework.model import BayesianMMM, TrendConfig, TrendType

cfg = ModelConfig(
    inference_method=InferenceMethod.BAYESIAN_NUMPYRO,
    n_draws=500, n_tune=500, n_chains=2,
    use_parametric_adstock=True, optim_seed=0,
)
mmm = BayesianMMM(panel, cfg, TrendConfig(type=TrendType.LINEAR))

What about all the priors — \( \beta \), \( \alpha \), \( \lambda \), seasonality, noise? The framework ships weakly-informative defaults for every one: wide enough not to bully the data, tight enough to keep the sampler in sane territory. Effect sizes are constrained positive — "ads don't destroy sales" is itself a prior! Workshop 01's job was to make you able to read such defaults, not to re-derive them every time; you can override any of them per channel, but today we take the defaults. The free parameters that get priors and posteriors: \( \beta \) per channel, adstock \( \alpha \) per channel, saturation \( \lambda \) per channel, intercept, trend slope, seasonality coefficients, a price coefficient, and noise \( \sigma \).

5 · Fit, then face the diagnostics gate

One call runs the whole workshop-02 machine — NUTS explores the joint posterior over all the parameters at once:

fit = mmm.fit(random_seed=0)   # sampled in 9s on the recorded run

Then — before reading any result — the diagnostics gate from workshop 02: (1) r-hat ≈ 1.00, meaning the independent chains agree with each other; (2) divergences = 0, meaning the sampler never broke; (3) bulk ESS comfortably in the hundreds, meaning enough effective draws to trust summaries.

Illustrative recreation (seeded) — what the notebook's beta_TV trace plot looks like when the gate passes: two chains wandering the same territory with no trends and no flat stuck stretches — workshop 02's "fuzzy caterpillar." The measured run's actual trace has exactly this character (r-hat 1.009 across 2 chains × 500 kept draws).

6 · Does the model reproduce the data?

First sanity check after any fit: push the posterior back through the model and compare against the observed KPI. Because the posterior is a distribution, the prediction is too — the framework draws many plausible KPI paths and reports the band containing 90% of them, not a single line pretending to certainty. On the recorded run: R² = 0.90, and the 90% predictive band covers 92% of weeks. That second number matters as much as the first — a 90% band that caught every week would be telling a different lie. The uncertainty here is calibrated, not decorative.

A strong fit — and a deliberately incomplete victory. A good fit does not prove correct attribution. The model could track sales beautifully while crediting the wrong channels: a model that mistakes seasonality for TV, or a demand-chasing budget for media effectiveness, can fit better than the truth. On real data this failure is invisible — which is precisely why this series practices on synthetic worlds, and why stress 00 exists: it manufactures worlds where every diagnostic is green and the attribution is silently wrong. Here, in the clean world, we can do what no real analyst can: open the answer key.

7 · The moment of truth: grading against the answer key

The model's headline output is each channel's contribution, computed by the counterfactual the term promises: predict sales with history as it happened, re-predict with one channel's spend forced to zero everywhere, and take the difference. The framework's compute_counterfactual_contributions does this per posterior draw, so each contribution arrives with a 90% HDI (highest density interval — the narrowest range holding 90% of the posterior, from workshop 00) instead of a bare number. The DGP computed the true contributions the same way — same question, same units — so the comparison is apples to apples. Unsealing the key:

contrib = mmm.compute_counterfactual_contributions(
    compute_uncertainty=True, hdi_prob=0.9, random_seed=0)

Measured — the baked notebook's recorded fit (seeded NumPyro run, random_seed=0). Dark bars are the sealed answer key; colored bars are the model's estimates with 90% HDI whiskers. Median |relative error| = 7%; truth inside the 90% HDI for 100% of channels. Toggle to "Relative error" to see each channel's miss with its HDI expressed in the same relative units.

Read the table and the chart together, the way you would brief a stakeholder:

• Every estimate lands close to its truth bar, and the relative-error column quantifies "close" per channel — no eyeballing required.
• Every whisker (the 90% HDI) brackets the truth. This is the part a point-estimate model cannot give you: the interval claims a range, and in a world where we can check, the claim holds.
• The ranking comes out right — TV > Search > Social > Display, with the top channel correctly identified — and ranking is what budget decisions actually consume.

This is what earned trust looks like: not "the model converged," but "on a world where the answer is checkable, the model finds it, and its stated uncertainty is honest." The stress series then asks the harder question — what happens when the world stops cooperating?

8 · What just happened, in Bayes terms

Strip away the marketing words and you have run exactly the workflow of workshops 00–02, at scale: (1) a generative story with a few dozen unknown parameters; (2) priors on all of them (the framework's weakly-informative defaults); (3) the likelihood of 156 observed weeks under any candidate parameter setting; (4) NUTS exploring the joint posterior — every kept draw is a complete parameter set, a full candidate explanation of three years of business history.

The contributions you just graded were never parameters at all. They are derived quantities: for each posterior draw, the model runs its zero-out counterfactual and gets one complete attribution story; section 7's table summarizes 1,000 such stories (2 chains × 500 draws). The chart below makes that concrete — each thin line is one draw's week-by-week claim about what TV contributed. The posterior is not a number with error bars bolted on; it is a population of full explanations, and every summary we report is a statistic of that population. That idea powers everything in workshop 05.

Illustrative recreation (seeded) — 40 thin lines, each one posterior draw's weekly TV-contribution path (drawn here by jittering \( \alpha, \lambda, \beta \) around the world's true values on the seeded illustrative spend), with the posterior mean and the true weekly path overlaid. In the notebook, the per-draw totals of these paths average to section 7's TV estimate exactly — contributions are statistics of draws.

One more thing the notebook's posterior explorer shows when you flip channels: some parameters are pinned down sharply while others stay wide in the same healthy fit — often because \( \beta \) and \( \lambda \) can partly trade off (a taller curve bent sooner can mimic a shorter curve bent later). The contribution stays well-determined even when the individual knobs share credit. Reading those shapes — widths, trade-offs, what the data did and didn't pin down — is exactly workshop 04's job.

9 · Glossary & takeaways

The glossary, now with scars — you have touched every one of these: