Aurora 04 · Reporting — putting it in front of the board

Chapter 4 of the Aurora Coffee Co. story. The analysis is done — the base MMM produced contributions, ROAS and marginal ROAS with credible intervals, and the extended models explained what TV is really doing. Now it has to land with people who don't read trace plots. mmm_framework renders a self-contained, themed HTML report straight from a fitted model: executive summary, ROAS with uncertainty, additive decomposition, diagnostics, and a causal-assumptions section so stakeholders see what the number rests on. This page mirrors nbs/aurora/04_reporting.ipynb — and because Aurora's world has a known ground truth, we can grade the report's own numbers.

What a decision-grade report must carry

A dashboard hands the board a point estimate. A posterior hands the analyst ten thousand draws. The report is the translation layer between them, and three things must survive the translation:

• Uncertainty intact. Every headline number — attributed revenue, blended ROI, per-channel ROAS — ships with its credible interval. A point without an interval is a guess wearing a suit.
• Diagnostics visible. Convergence (R-hat, ESS, divergences) is printed and flagged, not buried. If the fit was rushed, the report says so.
• Decomposition additive. Baseline + trend + seasonality + media + controls must sum to the model's predicted revenue, so every claim reconciles back to the top line. Strip one component and the story stops adding up — literally.

And one thing a report can never do for itself: know whether the numbers are causal. Good fit, tight intervals and green diagnostics are all compatible with confounding bias (the stress series measures exactly this). So the report ships a Causal Assumptions section that states, in stakeholder language, what identification rests on — and tells the reader to anchor high-stakes channels with experiments.

One call: the MMMReportGenerator API

The generator builds directly from a fitted BayesianMMM — it extracts contributions, ROAS, fit statistics and diagnostics for you. The notebook fits the Aurora panel (demo-sized: 2 chains × 400 draws, 20 seconds) and renders in two lines:

from mmm_framework.reporting import (
    MMMReportGenerator, ReportConfig, SectionConfig, ColorScheme, ColorPalette)

config = ReportConfig(
    title="Aurora Coffee Co. — Marketing Effectiveness",
    subtitle="Q3 Planning Review",
    client="Aurora Coffee Co.",
    analysis_period="2023 – 2024 (weekly)",
    color_scheme=ColorScheme.from_palette(ColorPalette.CORPORATE),
    confidential=True,
    diagnostics=SectionConfig(enabled=True),
)
report = MMMReportGenerator(model=mmm, panel=panel, results=results, config=config)
html = report.render()                          # the full HTML as a string
out  = report.to_html("artifacts/aurora_report.html")   # ...or written to disk

The baked run prints wrote artifacts/aurora_report.html (925,354 chars of HTML). One caveat the notebook flags: report charts load Plotly from a CDN, so rendering the file needs a network connection; it is otherwise fully self-contained — no server, no Python, just a file you can email to the CMO.

What's inside the report

The generated Aurora report contains these sections, in order (headings extracted from the baked artifact). The toggle shows the notebook's second build — the executive cut via the fluent ReportBuilder, which drops methodology and diagnostics for a shorter deck:

• Executive Summaryheadline stats with 80% CIs; key finding; "uncertainty matters" box
• Model Fitobserved vs posterior prediction with credible band
• Channel Performancedetailed ROI estimates — mean, 80% CI, evidence assessment
• Revenue Decompositionstacked components over time; sums to predicted revenue
• Saturation & Carryover Effectsresponse curves and adstock decay per channel
• Causal Assumptionsidentification strategy; robustness to unobserved confounding
• Methodologymodel specification, priors, inference settings
• Model DiagnosticsR-hat, ESS, divergences; trace plots; prior vs posterior

Measured — section headings extracted from the baked artifacts/aurora_report.html. The executive cut mirrors the notebook's ReportBuilder call: .enable_all_sections().disable_section("methodology") .disable_section("diagnostics").with_credible_interval(0.8). Note what the exec cut keeps: the intervals and the causal-assumptions section travel with every build.

from mmm_framework.reporting import ReportBuilder

exec_report = (ReportBuilder()
    .with_model(mmm, panel=panel, results=results)
    .with_title("Aurora — Executive Summary").with_client("Aurora Coffee Co.")
    .enable_all_sections().disable_section("methodology").disable_section("diagnostics")
    .with_credible_interval(0.8)
    .build())
exec_report.to_html("artifacts/aurora_exec_summary.html")

The report's own executive summary, from the baked artifact (which prints the revenue series in its native $-thousands units; converted to dollars here): marketing-attributed revenue $9.6M (80% CI [$1.1M – $22.0M]), blended marketing ROI 0.62 (80% CI [0.07 – 1.43]), marketing contribution 11.0% (80% CI [1.3% – 25.4%]) of $87.3M total revenue. Notice that the blended-ROI interval straddles 1.0 — the report's own headline admits the program might not pay for itself, which is exactly the kind of sentence a dashboard never produces.

The ROI table, recreated — and graded against the truth

The heart of the report is the Detailed ROI Estimates table: per-channel ROI with an 80% credible interval and a plain-language evidence assessment. Below is that table as a chart, with three views: how the report presents it, what's left after an executive strips the intervals, and how it grades against Aurora's known ground truth.

Measured — ROI means, 80% CIs and assessments extracted from the baked report artifact (nbs/artifacts/aurora_report.html, the file 04_reporting.ipynb writes); ground-truth ROAS from the Aurora generator (nbs/builders/aurora.py). Toggle the views and watch what each one would make you decide.

The graded view is the honest part of this page. The report — correctly, given what the base model can see — labels Search 1.04 [0.13, 2.22] "Uncertain" and TV 0.33 [0.05, 0.78] "Underperforming". The oracle says Search's true ROAS is 0.66 and TV's is 2.14. Search is a demand-chasing mirage (chapter 1, stress 03); TV works through awareness, which the base model structurally cannot credit (chapter 3). The generator cannot know any of that — no report generator can. What it can do is ship the caveat:

Theming & design tokens

Reports are branded with a ColorScheme built from named palettes or custom tokens. The notebook's measured output lists the five built-ins: ['SAGE', 'CORPORATE', 'WARM', 'MONOCHROME', 'AUGUR'] (AUGUR is the editorial "Media Performance Readout" theme). Every color in the report — header band, stat callouts, chart traces, borders — flows from one token set, so a rebrand is one argument, not a CSS hunt. Click a palette to restyle the preview:

Measured tokens, sketched layout — the palette hex values are the framework's actual ColorScheme.from_palette(...) constants (src/mmm_framework/reporting/config.py), and the numbers are the baked report's executive-summary and ROI figures. The mini-layout is a stylized sketch of the report header + stat-callout pattern, not the real template — open the live example for the genuine article. The notebook's report uses CORPORATE; the exec cut uses the warm palette.

"Just give me the number" — what you must not strip

Every analyst gets this request. The report is built so that complying honestly is possible — and complying dishonestly takes deliberate effort. The doctrine, in order of how expensive each omission is:

1. Never strip the interval. Social's point estimate is 0.71 — reads as "below break-even, cut it." Its 80% interval is [0.09, 1.69]: the data is consistent with Social returning nine cents or $1.69 per dollar. Those are opposite decisions. The honest sentence is a probability statement — \( P(\mathrm{ROAS}_{\text{Social}} > 1) \) — not a point. Interpreting Results covers how to read these; the stakeholder guide covers how to say them out loud.
2. Never strip the diagnostics line. One row — divergences, R-hat, ESS — is the difference between "the posterior is real" and "the sampler gave up." This demo's report flags all three. If you delete the row, you own the consequences.
3. Never strip the causal-assumptions section. It is the only part of the report that says what the numbers mean. The report's badges are statements about posterior precision, not identification — TV's "Underperforming" badge has the truth (2.14) sitting far above its interval [0.05, 0.78] because the bias is structural, not statistical.
4. Keep the decomposition additive. Components sum to predicted revenue by construction. The moment someone hand-edits one bar "for the narrative," the report stops reconciling and every other number becomes negotiable.

See a real generated report

The notebook itself writes two more: artifacts/aurora_report.html (the full corporate-palette build) and artifacts/aurora_exec_summary.html (the exec cut) — both regenerate when you run it.