Aurora 03 · Extended Models — What is TV really doing?

Chapter 3 of the Aurora Coffee Co. story. The base MMM handed us two unsolved mysteries: TV and Display look nearly worthless, and the two products — Original and Cold Brew — seem to move together in ways spend can't explain. This chapter answers both with the framework's extended models: a NestedMMM that models the TV → awareness → sales pathway explicitly, and a MultivariateMMM that models both products at once with a cross-effect between them. Because Aurora's world is synthetic with known ground truth, every claim on this page is graded against the real answer — including one claim that looks confirmed and turns out to be only weakly identified.

The blind spot, quantified

In Aurora's true data-generating process, TV and Display barely touch sales directly. They build a slow brand-awareness stock, and the awareness stock sells coffee. The base model has no awareness variable, so it can only credit the (tiny) direct slice — and calls the brand channels weak. The gap is not subtle:

Measured — base-model ROAS from the chapter-2 fit (nbs/aurora/02_base_mmm.ipynb) vs the known true ROAS from the Aurora generator (nbs/builders/aurora.py). TV: 0.35 vs 2.14. Display: 0.67 vs 2.11. Meanwhile Search — the demand-chasing mirage from chapter 0 — reads 1.05 against a truth of 0.66. The base model undervalues the brand engines three- to six-fold and overvalues the demand-chaser by ~60%.

Part A — Mediation: TV → Awareness → Sales

Aurora measures awareness in a monthly brand-tracker survey — 26 noisy observations across 104 weeks. The NestedMMM treats awareness as a partially observed mediator: TV and Display feed it, it feeds sales, and the model decomposes each brand channel's effect into direct vs indirect (via awareness).

from mmm_framework.mmm_extensions.models import NestedMMM
from mmm_framework.mmm_extensions.builders import MediatorConfigBuilder, NestedModelConfigBuilder

nested_cfg = (NestedModelConfigBuilder()
    .add_mediator(MediatorConfigBuilder("awareness")
                  .partially_observed(observation_noise=0.1)   # the monthly survey
                  .with_positive_media_effect(sigma=1.0)
                  .with_direct_effect(sigma=0.5).build())
    .map_channels_to_mediator("awareness", ["TV", "Display"])  # brand channels build awareness
    .build())

nested = NestedMMM(X, aurora.sales_total, list(CHANNELS), nested_cfg,
                   mediator_data={"awareness": aurora.awareness_survey}, index=aurora.weeks)
nested.fit(draws=500, tune=500, chains=2, cores=1, random_seed=0)
med = nested.get_mediation_effects()

The decomposition is the reveal. Most of TV's and Display's effect flows through the awareness pathway the base model couldn't see rather than reaching sales directly — though on PyMC 6 the model only partially recovers how large that mediated share is:

Measured (PyMC 5) — get_mediation_effects() from the baked notebook: TV direct 0.05 vs indirect 151.48 (total 151.53); Display direct 0.11 vs indirect 186.28 (total 186.39), a near-total proportion_mediated = 1.00 for both channels. On PyMC 6 the mediated share only partially recovers — proportion_mediated ≈ 0.69 (TV) and ≈ 0.40 (Display) — so the direct slice is no longer negligible; the awareness pathway is still the dominant route for TV, and the wide intervals are the honest signal. Search and Social were not mapped to the mediator (they are pure direct-response channels in Aurora's world), so the decomposition applies only to the brand channels.

Is that reveal real, or an artifact of bolting on a mediator? Because this world is synthetic, we know the true mediated share — and on PyMC 6 the model only partially recovers it:

Measured — the model's proportion_mediated vs the generator's true_mediated_share. On PyMC 5 the model read 1.00 for both (vs a truth of 0.99 TV / 0.97 Display); on PyMC 6 it only partially recovers — ≈ 0.69 (TV) and ≈ 0.40 (Display) — beating the base model's blind spot but leaving the mediated channel under-recovered with wide intervals. Search and Social are truly unmediated (0.00) and were not routed through awareness in the model.

The mediator itself is recovered too. From just 26 monthly survey points, the model's latent awareness path tracks both the survey and the normally-hidden true awareness curve:

Illustrative recreation (seeded) — the baked notebook's chart shows the model's latent-awareness posterior (mean + 90% band, z-scored) tracking the sparse monthly survey dots and the true awareness curve across all 104 weeks. This recreation reproduces that qualitative result with the same structure: a slow awareness stock, monthly noisy observations of it, and a smoothed estimate with a band. The measured claim it stands in for: the NestedMMM's latent path follows the truth even between survey points.

Interactive: how mediation hides a channel from a base model

Why does routing an effect through awareness make it invisible to a spend-on-sales regression? Timing. A direct effect arrives within a few weeks of the spend that caused it — adstock can see that. A mediated effect first accumulates into a slow awareness stock, so the sales lift decouples from the spend pulses that built it. Drag the slider to move a fixed-size TV effect between the two paths and watch the spend↔effect correlation — the statistical handle a base model grips — fall away.

Illustrative — a toy simulation computed live in your browser, not Aurora's fitted model. Flighted TV spend (grey) drives a direct adstocked path and a slow awareness stock (decay 0.92/week); the slider splits a fixed total effect between them. At 0% mediation the effect hugs the spend flights; at ≈100% (Aurora's measured case) the effect is a smooth wave with almost no same-week relationship to spend — which is why the base model handed TV's credit to the trend and the demand-chasing channels instead.

Part B — Cannibalization: do the two products fight?

Aurora sells two products, and the Cold Brew launch looked like it ate Original's summer sales. A MultivariateMMM models both outcomes at once: shared media, a cross-effect ψ (Cold Brew → Original) built with the cannibalization_effect helper, and an LKJ-correlated residual structure so a shared demand shock can hit both products.

from mmm_framework.mmm_extensions.models import MultivariateMMM
from mmm_framework.mmm_extensions.builders import (
    MultivariateModelConfigBuilder, OutcomeConfigBuilder, cannibalization_effect)

mv_cfg = (MultivariateModelConfigBuilder()
    .add_outcome(OutcomeConfigBuilder("sales_original", column="sales_original")
                 .with_positive_media_effects(sigma=0.5).build())
    .add_outcome(OutcomeConfigBuilder("sales_coldbrew", column="sales_coldbrew")
                 .with_positive_media_effects(sigma=0.5).build())
    .add_cross_effect(cannibalization_effect(source="sales_coldbrew", target="sales_original"))
    .build())

mv = MultivariateMMM(X, outcomes, list(CHANNELS), mv_cfg, index=aurora.weeks)
mv.fit(draws=500, tune=500, chains=2, cores=1, random_seed=0)
ce = mv.get_cross_effects_summary()

The cross-effect summary comes back looking decisive: on PyMC 6 the posterior mean is ≈ −1.13, sitting entirely below zero (on PyMC 5 the same fit returned a near-zero −0.0003, 94% HDI [−0.0008, −0.0000]). The naive read writes itself: "confirmed: Cold Brew cannibalizes Original." Hold that thought.

Illustrative recreation (seeded) of PyMC 5 measured values — the curves are seeded recreations shaped to the baked notebook's PyMC-5 summaries: the prior is the model's actual \( \psi = -\,\mathrm{HalfNormal}(0.3) \) density (prior mean magnitude 0.236, sd 0.176); the plotted posterior matches the PyMC-5 mean −0.0003 and 94% HDI [−0.0008, −0.0000], a needle at zero. On PyMC 6 this reverses: the posterior does not pin at zero — it relocates to a sizeable negative value (mean ≈ −1.13, shifted ≈ 5 prior-sds) and comes back wider than the prior. The mass is still entirely below zero, but ψ is now weakly identified: trust the sign, not the magnitude.

Is this real, or did the prior decide it for us?

The cannibalization prior is one-sided by construction: \( \psi = -\,\mathrm{HalfNormal}(\sigma) \), so \( \psi \le 0 \) with probability one before any data arrive. That makes "the interval is below zero" / \( P(\psi < 0) \approx 1 \) nearly vacuous — it can simply restate the prior. The honest replacement is the framework's parameter-learning diagnostic: contraction \( c = 1 - \mathrm{Var}_{\text{post}} / \mathrm{Var}_{\text{prior}} \) (→ 1 means the data pinned the parameter) and the prior↔posterior overlap coefficient (→ 0 means strong learning).

Where the co-movement actually lives: the shared demand wave

The two products genuinely do move together — and the model registers it through two channels for linkage. After media is accounted for, the outcome residuals remain positively correlated at +0.41 (measured): Aurora's shared demand wave lifts Original and Cold Brew at the same time. The substitution baked into Aurora's generator is only weakly identified as a direct cross-effect by this two-outcome model — its sign, not its magnitude (see above) — and it also surfaces as this shared-demand co-movement.

Illustrative recreation (seeded), at the measured correlation — two residual series constructed to share a common demand wave at exactly the baked notebook's measured residual correlation, r = +0.41 (the notebook draws this as the get_correlation_matrix() heatmap). The practical consequence is real either way: the products must be planned together, because a demand swing moves both at once — just not because Cold Brew is directly eating Original.

The caveat: extensions add structure the data cannot always identify

The discipline this chapter modeled: never accept a sign statement from a one-sided prior; demand contraction and overlap before believing any extension-specific parameter; and validate the mediation share against external evidence (a survey, a lift test — see chapter 1 for experiment calibration) rather than the model's own fit.

Wrap-up

Extended models also serialize like the base model (nested.save("artifacts/aurora_nested") / NestedMMM.load(...) — trace restored, measured in the notebook). Aurora now has everything: a causal map, experiment anchoring, brand-pathway value, and product interactions. Next: turning it into a board deck in chapter 4 · Reporting, and into money in chapter 5 · the Unified Workflow.