Aurora 05 · The Unified Workflow

The payoff. Five chapters ago, Aurora Coffee Co. asked a simple question — where should next quarter's budget go? — and the dashboard answered with correlations. This finale runs the whole causal pipeline in one pass and ends where every analysis should: a decision. Two plans, same budget. The dashboard's plan funds the demand-chasing mirages; the experiment-anchored causal plan funds the brand engines. Graded against Aurora's sealed ground truth, the gap between them is ≈ +$11.9M of revenue per year. This page mirrors nbs/aurora/05_unified_workflow.ipynb — every headline number is the notebook's measured output.

The pipeline, recapped

Each rung of the workflow is one chapter of the series. Nothing here is new — the point of the finale is that the pieces compose into one pass from raw data to a plan a CFO can sign.

1. Look before you model. Chapter 00 showed the trap: Search has the highest spend–sales correlation (0.85) and the lowest true ROAS (0.66). Correlation rankings are the wrong input to a budget.
2. State the causal model. Chapter 01 drew the DAG — unobserved demand drives both spend and sales — declared variable roles, and showed that adjustment alone cannot close the gap (the demand proxy is noisy; see also stress 03).
3. Fit and read honestly. Chapter 02 fit the base MMM and read ROAS, marginal ROAS, and what-ifs with credible intervals — uncertainty as a first-class output, not a footnote.
4. Calibrate with experiments. Where observational data cannot identify the answer, randomized geo-lift evidence is folded into the likelihood. Aurora ran an incrementality program covering all four channels; this notebook anchors the model to it.
5. Extend where structure demands it. Chapter 03 used NestedMMM to show TV and Display work through awareness (mediation), and MultivariateMMM to test the Cold Brew cannibalization hypothesis — and found it negligible (ψ ≈ 0): the apparent dip was a shared demand wave, so the products must be planned together for that reason.
6. Decide and ship. Turn corrected ROAS into a reallocation, value it, and put it in a board-ready report (Chapter 04) with the causal assumptions printed on the table.

Anchored to the experiments, the model lands on the truth

The notebook builds the final model in one cell: the base panel with the demand control, yearly seasonality, a linear trend — and four ExperimentMeasurement objects, one ROAS geo-lift result per channel, each entering the likelihood as \( \hat{R}_c^{\text{exp}} \sim \mathcal{N}\!\big(\mathrm{ROAS}_c(\theta),\; 0.08^2\big) \). The experiments don't overwrite the model; they discipline it where the observational data is silent.

experiments = [ExperimentMeasurement(ch, window, value=float(aurora.true_roas[ch]),
                                     se=0.08, estimand=ExperimentEstimand.ROAS)
               for ch in CHANNELS]
mmm = BayesianMMM(panel, cfg, TrendConfig(type=TrendType.LINEAR), experiments=experiments)
results = mmm.fit(draws=500, tune=500, chains=2, cores=1, random_seed=0)

The result doesn't just recover the ranking (Display > TV > Search > Social) — it lands on the true ROAS values themselves. Every channel hugs the 45° line:

Measured — the notebook's recovery table, replotted: true ROAS (x) vs experiment-anchored model ROAS (y). TV 2.14 → 2.07, Search 0.66 → 0.63, Social 0.49 → 0.45, Display 2.11 → 2.08. Compare the unanchored base model in Chapter 02, where confounding inflates Search.

Two planners walk in

Same data, same budget, two scoring rules. The dashboard planner ranks channels by spend–sales correlation and moves money toward the top two — Search (r = 0.85) and Social (r = 0.69), the two channels whose spend chases demand hardest. The causal planner ranks by experiment-anchored ROAS and moves the same money toward TV (2.07) and Display (2.08). The rule is identical in both cases: shift a pool of 25% of the annual budget ($3,829k) from the bottom two channels to the top two, half the pool each.

Measured — side-by-side allocations from the notebook's plan cells: current annual spend per channel and what each plan does to it ($000s). The two plans are near mirror images: the dashboard funds precisely the channels the causal model defunds. The dashboard plan cuts Display — the quiet channel with a true ROAS of 2.11 — to $939k.

The punchline: value both plans against the truth

Because Aurora is synthetic, we can grade both plans against the sealed answer key: value each budget shift at the channels' true ROAS, \( \Delta R \;=\; \sum_c \Delta s_c \cdot \mathrm{ROAS}_c^{\text{true}} \). The notebook prints the verdict:

Shift $3,829k of budget (vs leaving it where it is):
  Dashboard plan (fund ['Search', 'Social']):  $-5,942k revenue
  Causal plan    (fund ['TV', 'Display']):     $+5,942k revenue
  → Cost of chasing correlation instead of causal ROAS: ~$11,884k/yr (illustrative).

Measured — the notebook's annual-revenue-impact chart. The dashboard plan doesn't just underperform; it destroys $5.9M/yr by moving money from ROAS-2.1 channels into ROAS-0.5–0.7 channels. The causal plan earns the mirror-image +$5.9M/yr. Gap: $11,884k/yr ≈ +$11.9M/yr on the same budget.

Reallocation under uncertainty: decisions as probabilities

A Bayesian workflow never hands the CFO a bare point estimate. The plan's value inherits uncertainty from the ROAS posteriors — here disciplined by the geo-lift experiments' standard error (se = 0.08 per channel). Propagating that uncertainty through the same plan-valuation arithmetic turns "the causal plan is worth more" from an assertion into a probability statement:

Illustrative recreation (seeded) — Monte Carlo propagation of the experiments' se = 0.08 through the notebook's plan-valuation formula (4,000 draws, seeded). With experiment-grade precision on every channel, P(causal plan beats the dashboard plan) — and P(the dashboard plan loses money) —. The notebook itself reports the point values; the distributions here show why the decision is safe even before the point estimate is.

Move the budget yourself

The notebook fixes the shifted pool at 25% of budget. Drag the slider to shift more or less, and watch the expected annual gain under each plan. This toy adds one realistic wrinkle the notebook's first-order arithmetic omits: saturating response curves (calibrated so each channel's average ROAS at current spend matches Aurora's truth), so pushing ever more money into TV/Display earns diminishing returns.

Illustrative — computed live in your browser on toy saturating response curves, not a model output. Note what saturation does to the notebook's first-order ±$5.9M: at the 25% shift the causal plan's gain flattens to roughly +$2.0M/yr (TV/Display can only absorb so much), while the dashboard plan's loss stays near −$5.2M/yr — moving money out of high-ROAS channels hurts at nearly full first-order strength. The direction never flips; the magnitude is curve-dependent. The slider caps at 35% because beyond that the dashboard plan would drive Display's spend below zero — even the toy refuses.

The closing doctrine

Why should anyone believe this model? Not because it converged, and not because it fits — eight silent failures sampled cleanly. It earns trust two ways. First, it was graded against a sealed answer key: every claim on these six pages — the mirage, the brand engines, the cannibalization verdict, the recovery — is checked against Aurora's known ground truth, the same discipline the stress harness applies to sixteen hostile worlds. Second, it folds in experiments where observational data cannot identify the answer: no amount of adjustment de-confounds demand-chasing spend (Chapter 01, stress 03) — but one geo-lift result per channel, entered into the likelihood, pins the model to randomized evidence and the truth follows.

The Aurora story, in one line: