Workshop 04 — Reading the Posterior

Workshop 03 ended with a bar chart: one number per channel. But the model didn't give you a bar chart — it gave you a thousand complete, internally-consistent versions of reality, each one a full set of parameter values that could plausibly have produced your data. The bar chart was just one way of squashing them. This guide teaches you to use all of them: honest interval summaries (HDI), the forest plot, the joint correlations that make mixing point estimates illegal, a check that the data actually taught you something, and the two charts every stakeholder sees — fit and decomposition — with their uncertainty kept on.

Same world, same fit as workshop 03: the synthetic "clean" world from src/mmm_framework/synth/dgp.py — four media channels (TV, Search, Social, Display) driving 156 weeks of sales with carryover, diminishing returns, trend, seasonality, and one control — fit with NUTS via numpyro at 500 draws × 2 chains. One fit; everything on this page is reading, not refitting. And because the world is synthetic, the true causal contribution of every channel is the sealed answer key we grade against (TV 5,668 · Search 4,531 · Social 4,017 · Display 2,958 sales units).

The posterior is a TABLE of worlds

The fit's output is, conceptually, a big table. Each column is a parameter of the model — a channel's effect size beta_TV, its carryover rate adstock_alpha_TV, its diminishing-returns curvature sat_lam_TV, the trend slope. Each row is one draw (also called a sample) — one complete, self-consistent setting of every parameter at once, kept by the MCMC sampler from workshop 02 because it explains the data plausibly. Two chains × 500 draws = 1,000 rows. Here are the first five, verbatim from the fit:

Read one row across: "maybe TV's effect is this big, with this much carryover and this much curvature, while Search's effect is that big and the trend climbs at this rate." That entire sentence — all parameters together — is one draw. This is the single most important mental model in the series:

Everything else on this page is just different ways of looking at this table: one column at a time (a marginal — the distribution of a single parameter, ignoring the others), two columns at a time (a joint view), or whole-row computations (workshop 05).

Summaries done right: mean, median, and the HDI

A thousand numbers per parameter is too many for a slide, so we summarize. Two decisions hide inside every summary. Center: the mean (average of the draws) and the median (the middle draw) agree when the marginal is symmetric; when it's skewed — a long tail on one side, common for effect sizes forced positive — the mean gets dragged toward the tail. Interval: two ways to build a "90% range":

• the equal-tailed interval (ETI) — chop 5% off each end (the 5th to 95th percentile). Simple, but on a skewed marginal it can exclude values more plausible than ones it includes;
• the HDI (highest-density interval) — the narrowest range holding 90% of the draws: every value inside is more plausible than every value outside. The framework's default, and the one to report.

The notebook hunts for the most-skewed marginal programmatically and finds beta_Search (skewness +2.14). Its mean is 1.426 vs median 1.343 — the tail drags the mean. Its 90% HDI is (0.991, 1.868), width 0.878; the 90% ETI is (1.055, 2.018), width 0.964. Drag the slider to ask for more or less certainty and watch both intervals respond — the HDI is computed live on the fit's actual density curve.

Measured — the density curve is the actual beta_Search posterior KDE extracted from the baked notebook; the HDI (shaded) and ETI (dotted) are computed live in your browser on that curve — the same grid math as the notebook. At 90% the notebook measures HDI (0.991, 1.868) vs ETI (1.055, 2.018): the ETI's left edge excludes a sliver near the peak — values among the most plausible in the whole posterior — while its right edge includes thin tail. The HDI fixes both by construction. Notebook assert: HDI widths grow monotonically — 0.31 / 0.64 / 0.88 / 1.09 at 50/80/90/95%.

One habit to build now: say "90% credible interval", and mean it literally — given the model and the data, there's a 90% chance the parameter is in this range. That plain-English reading is the one Bayesian intervals actually license (workshop 00).

The standard one-line-per-parameter summary is az.summary. For the four channel betas (measured, 90% HDI):

The table carries center, spread, interval, and the workshop-02 health checks (ess_bulk/ess_tail — how many independent draws your correlated draws are worth; r_hat — chain agreement, want ≈ 1.00) for every parameter. Note the betas are in the model's scaled units, not sales dollars — fine for comparing health, misleading for comparing importance. Dollar-scale contributions arrive in the decomposition below, and become decisions in workshop 05.

The forest plot: the posterior at a glance

A forest plot draws each parameter as one horizontal line: a dot at the center (here, the median), a thick line spanning the 50% HDI, a thin line spanning the 90% HDI — the summary table turned into a picture. The rule for reading it: compare lines, not dots. If two channels' intervals overlap heavily, the posterior considers either ordering plausible — "TV beats Social" is then a coin-flip statement dressed up as a finding.

Measured — medians and HDIs from the baked fit (TV median 2.37, Search 1.34, Social 2.68, Display 1.41; hover for each channel's 50% and 90% HDIs). Every 90% interval is comfortably above zero — the model is confident each channel does something — yet all 6 of 6 channel pairs overlap at 90%: the posterior refuses to fully rank the channels, and that's a finding, not a failure.

This is the forest plot's quiet superpower: it makes overclaiming visually impossible. A bar chart of posterior means would happily sort the channels and invite a ranking story; the forest shows the intervals leaning on each other, so "we can't tell these two apart yet" is right there on the slide. With many parameters, az.plot_forest(idata, var_names=...) builds this in one line.

Joint structure: the part point estimates hide

Marginals (one column of the table at a time) throw away how parameters move together across worlds. A channel's modeled response is roughly effect size × shape of diminishing returns: \( \text{response} \approx \beta \cdot f_\lambda(\text{adstocked spend}) \). Many different \( (\beta, \lambda) \) pairs produce nearly the same fitted curve over the spends we observed — so the data can pin down the product tightly while leaving each ingredient loose. The posterior expresses that as correlation between the two parameters across draws. Measured, per channel: corr(β, λ) = TV −0.82 · Search −0.74 · Social −0.84 · Display −0.74. Here are the actual draws for Social, the strongest trade-off:

Measured — 350 of the 1,000 real posterior draws for Social, subsampled to keep the page light (the subsample preserves the full-posterior correlation, r = −0.84). Worlds with a sharper curve (higher λ) compensate with a bigger effect size (higher β). The black dot is the posterior mean — inside the cloud. The red × is the "mix-and-match" point: each parameter's own 80th percentile, taken separately. The measured share of draws with both parameters above their own 80th percentile is 0.0% — independence would give 4.0%. The red × is a world your own posterior says doesn't exist.

Why this matters for every number you report: the marginals are wide, but the thing the data actually pins down — the channel's total contribution, a per-draw product of β with its curve — is much tighter. Measured coefficients of variation (CV — spread relative to the mean):

Each channel's β wobbles a lot across worlds, but the contribution — computed inside each world, where β and its curve are consistent — wobbles far less. The posterior is unsure how to split credit between effect size and curvature, but quite sure about what the channel delivered. The trade-off cancels only if you stay inside a row. That hands us the golden rule of the whole series, and the engine of workshop 05:

Did the data actually teach us anything? Prior → posterior

A posterior is always prior × data. So before you report a posterior number, you owe yourself one check: how different is it from the prior you started with? If prior ≈ posterior, the data didn't speak — and the number you're about to report is your assumption wearing a lab coat. The framework ships a real diagnostic: mmm.compute_parameter_learning(). For every parameter it draws fresh prior samples, compares them with the posterior, and reports contraction = \( 1 - \sigma^2_{\text{post}} / \sigma^2_{\text{prior}} \) (near 1: the data pinned the parameter down; near 0: nothing learned; negative: a prior–data conflict worth investigating), an overlap score, a shift_z (how far the center moved, in prior standard deviations — a posterior can move without narrowing, and that's still learning), and a plain-English verdict per parameter.

Measured — contraction for all 20 parameters of this fit, sorted least-learned first, colored by verdict. Structural parameters the data sees every week are pinned hard (sigma 0.99, the control 0.99, seasonality ≈ 0.98, trend slope 0.96, intercept 0.92) — while the channel betas span the whole range: beta_Search 0.90 (strong), beta_Display 0.59 (strong), beta_TV 0.46 (moderate), and beta_Social 0.08 — relocated: its center moved +1.5 prior-sd, but it barely narrowed.

See it directly: prior vs posterior for each channel's β, with its measured contraction. Search's posterior is dramatically narrower than its prior (post sd / prior sd = 0.32); Social's barely narrowed (0.96) — though its center shifted (+1.5 prior sds), which is the only learning it got: exactly the pattern the relocated verdict names.

Measured centers and widths, illustrative shapes — the prior/posterior means and standard deviations (and the contraction labels) are the notebook's measured values from compute_parameter_learning; the curves are Gaussian-rendered from those moments for display. Search: the data spoke loudly. Social: the posterior barely narrowed (contraction 0.08) — but its whole distribution slid right, the relocated signature.

The posterior predictive check: can the model fake your data?

So far we've read parameters. The posterior predictive flips the direction: take each world in the table, simulate the KPI it implies (noise and all), and stack the simulations. If the model is adequate, your observed sales should look like just another simulation — nothing about it should surprise the model. mmm.predict() does this in one call.

Measured — the real observed sales (dots), the posterior-predictive mean and 90% band, and three of the fit's simulated KPI paths ("spaghetti" — each thin line is one posterior world, noise included), all extracted from the baked notebook. The band covers 92% of weeks (target ≈ 90%) with R² 0.90 — this is what a PASS looks like: dots threading the band, coverage roughly what was promised, spaghetti honestly data-shaped rather than suspiciously smooth.

The check has teeth in both directions: a band that misses 30% of weeks means the model is overconfident; a band that swallows everything with huge margins means it has learned almost nothing. Failures look like dots drifting outside the band in runs (a missing trend or seasonality), a band systematically too narrow in high-sales seasons (wrong noise model), or whole stretches the simulations never reach (a structural break).

Decomposition with uncertainty: where the sales came from

The business question behind every MMM: of the sales we observed, what part did each thing cause? The decomposition answers it by splitting the model's predicted KPI into additive layers — baseline (intercept), trend, seasonality, controls, and each media channel. mmm.compute_component_decomposition() returns the posterior-mean path of every component:

Measured — the real posterior-mean decomposition layers and observed sales from the baked fit. Notebook asserts: the layers sum to the model's prediction (corr 0.9999 with the posterior-predictive mean), and the media layers agree with the per-draw channel_contributions posterior used in the joint section — one consistent accounting throughout.

The stacked picture is what stakeholders ask for — and it's the most dangerous chart in MMM, because it shows means with no widths. Each colored layer is the average across 1,000 worlds that genuinely disagree. So the notebook puts the uncertainty back: for each channel it takes the weekly contribution in every posterior world, draws the 90% band around the mean path — and, because this world is synthetic, overlays the true weekly contribution path, the answer key a real analyst never gets to see. The measured result:

Measured — the share of weeks where each channel's true weekly contribution path lies inside the model's 90% band: TV 100% · Search 88% · Social 97% · Display 100%. On this clean world the bands are honest — the uncertainty was real information, not decoration. (On worlds engineered to break assumptions, that's exactly what fails first: see pressure testing.)

Glossary, and where this goes next