Reading Model Results
A Bayesian marketing mix model produces probability distributions, not single numbers. This is a feature: it tells you not just what the model thinks, but how confident it is.
The Key Shift in Thinking
Traditional models give you “TV ROI = 1.42.” A Bayesian model gives you “TV ROI is most likely between 1.1 and 1.8, with a best estimate of 1.4.” The second statement is more honest and more useful for decision-making.
Key Terms You Need to Know
Before diving into results, familiarize yourself with these concepts. You do not need statistical training to use them.
Channel ROI
ROI is the most important output for budget decisions. The model provides it as a distribution, not a single number.
How to Read an ROI Result
What this means: For every $1 spent on TV, the model estimates you earned $1.40 in incremental revenue. The true value is very likely between $1.10 and $1.80. We are confident TV is profitable (the entire range is above 1.0).
Contrast: An Uncertain Result
What this means: The best estimate is $1.10 for every $1 spent, but this could be as low as $0.40 (unprofitable) or as high as $2.30. The model cannot confidently determine whether display is profitable. Budget decisions should be made cautiously, and experimental validation is recommended.
Entire CI above 1.0. Safe to invest confidently.
CI spans 1.0. Needs validation before big changes.
Entire CI below 1.0. Consider reducing or testing.
Revenue Decomposition
The decomposition chart shows what drives your revenue. It breaks total sales into contributions from each media channel, base demand, and other factors.
Reading the Decomposition
Base is what you would sell with zero marketing. Seasonality captures predictable peaks and valleys. Media channels show incremental contribution above base. The uncertainty band around each shows the range of plausible values.
Common Misinterpretation
A channel contributing 5% of revenue does not mean it is unimportant. If it costs 3% of your budget and returns 5% of revenue, its ROI is strong. Always evaluate contribution relative to spend, not in isolation.
Saturation Curves
Saturation curves show the relationship between spend and incremental response. They tell you where you are on the curve of diminishing returns.
- Operating on the steep part of the curve
- Additional spend yields meaningful incremental returns
- Room to grow—consider increasing investment
- Example: Paid search at 40% saturation
- Operating on the flat part of the curve
- Additional spend yields minimal incremental returns
- Consider reallocating to under-saturated channels
- Example: TV at 85% saturation
Understanding Uncertainty
Uncertainty is the most valuable output of a Bayesian model. It tells you where you can act confidently and where you need more information.
Why Uncertainty Matters for Decisions
Narrow Intervals = High Confidence
When the model says TV ROI is 1.4 (1.2–1.6), the range is tight. You can make budget decisions with confidence. The data strongly supports this estimate.
Wide Intervals = Low Confidence
When the model says display ROI is 1.1 (0.4–2.3), the range is wide. The data does not strongly constrain this estimate. Major budget changes based on this estimate carry risk. This is where experiments add the most value.
Intervals Spanning 1.0 = Uncertain Profitability
When the 90% credible interval includes values both above and below 1.0, the model cannot confidently determine whether the channel is profitable. This is an honest answer that should drive experimentation, not frustration.
The Decision Framework
Use uncertainty to categorize channels: (1) confident and positive—invest; (2) confident and negative—reduce; (3) uncertain—test with experiments before making large changes. This framework prevents the two biggest mistakes: over-investing based on false precision, and under-investing because a single estimate looked low.
For Media Planners
Budget Allocation
The model provides the information you need for budget allocation, but with honest uncertainty about the precision of each estimate.
Step 1: Rank Channels by ROI Confidence
Start by categorizing channels using the traffic-light framework above. Channels where the entire 90% CI is above 1.0 are your foundation. Channels where the CI spans 1.0 need caution.
Step 2: Check Saturation Levels
A channel with high ROI but high saturation has less room to grow than a channel with moderate ROI and low saturation. The marginal ROI at your current spend level is more relevant than the average ROI.
Example: Budget Allocation Decision
| Channel | ROI (90% CI) | Saturation | Recommendation |
|---|---|---|---|
| TV | 1.4 (1.1–1.8) | 75% | Maintain or small increase |
| Paid Search | 2.1 (1.6–2.8) | 45% | Increase investment |
| Display | 1.1 (0.4–2.3) | 60% | Hold and test |
| Radio | 0.7 (0.3–1.1) | 80% | Consider reducing |
Optimization with Uncertainty
The framework can optimize budget allocation across channels, but the optimization accounts for uncertainty. Rather than a single “optimal” allocation, you get a range of good allocations.
Beware of Point-Estimate Optimization
An optimizer using only point estimates will concentrate budget in whichever channel has the highest estimated marginal ROI. But if that estimate is uncertain, the “optimal” allocation could be far from optimal. The MMM Framework’s optimization propagates uncertainty through the optimization, producing robust recommendations.
Flighting Decisions
Adstock (carryover) estimates inform media timing. Channels with longer carryover benefit from sustained presence; channels with short carryover benefit from concentrated bursts.
TV: Long Carryover (6–8 weeks typical)
Effects persist for weeks after airing. Sustained flight patterns maintain steady awareness. Gaps in TV advertising take weeks to recover from.
Digital: Short Carryover (1–3 weeks typical)
Effects are more immediate but fade quickly. Concentrated bursts around key periods (promotions, holidays) can be effective. Always-on at lower levels maintains baseline presence.
For CMOs
The Executive Summary
As a CMO, you need three things from a model: (1) which channels are working, (2) how confident we are, and (3) what to do about it.
What a Good Executive Summary Looks Like
Media Effectiveness Summary — Q4 2025
High confidence findings:
- TV is profitable with ROI of 1.4 (90% CI: 1.1–1.8). This is consistent with our Q3 geo lift test (1.1–1.5).
- Paid Search has the highest ROI at 2.1 (90% CI: 1.6–2.8) and is under-saturated at 45%.
Areas of uncertainty:
- Display ROI is uncertain (0.4–2.3). We recommend a geo test in Q1 to resolve this.
- Radio appears marginally effective (0.3–1.1). Given high saturation, consider reducing by 20% and measuring impact.
Recommendation: Shift 15% of Radio budget to Paid Search. Hold Display pending Q1 test results. Maintain TV at current levels.
Confidence Levels and What They Mean
Not all model outputs deserve equal trust. Use this framework to weigh recommendations appropriately.
Validated Estimates (Highest Confidence)
The model estimate is consistent with experimental results (geo lift tests, RCTs). Act on these with confidence. Example: TV ROI model estimate (1.1–1.8) overlaps with geo test (1.1–1.5).
Narrow, Unvalidated Estimates (Moderate Confidence)
The model is precise (narrow CI) but has not been validated experimentally. These are reasonable for moderate budget changes. Design validation tests for these channels.
Wide, Unvalidated Estimates (Low Confidence)
The model is uncertain (wide CI) and unvalidated. Avoid major budget changes based on these. Prioritize experiments here—this is where testing has the highest information value.
When to Act vs. When to Test
| Situation | Action | Rationale |
|---|---|---|
| ROI confidently above 1.0, channel under-saturated | Increase investment | Strong evidence of profitability with room to grow |
| ROI confidently below 1.0 | Reduce investment | Strong evidence of unprofitability |
| ROI uncertain (CI spans 1.0), large spend | Design experiment | High information value from resolving uncertainty on a large budget line |
| ROI uncertain (CI spans 1.0), small spend | Maintain current levels | Low risk from maintaining; experimentation may not be cost-effective |
| Year-over-year ROI changed dramatically | Investigate before acting | Could be genuine market shift, data issue, or modeling artifact |
The CMO’s Competitive Advantage
Organizations that combine model-based estimates with experimental validation build a compounding knowledge advantage. Each experiment improves model calibration. Better-calibrated models produce more confident estimates. More confident estimates enable bolder, higher-ROI budget decisions. This virtuous cycle is the genuine competitive advantage of rigorous measurement.
For the operating mechanics of that cycle — how to choose which experiments to run, how to feed their results back into the next MMM, and how the math underwrites the “compounding” claim — see the Closed-Loop Measurement & Calibration guide.
Common Questions About Results
Wide intervals mean the data does not strongly constrain the estimate. This happens when: (1) the channel has limited spend variation, (2) the channel is correlated with another driver, or (3) the sample size is small. The intervals are wide because the truth is genuinely uncertain—narrower intervals from other methods are often falsely precise.
Yes, in certain situations. Over-saturated channels can have negative marginal returns. Poorly targeted or poorly timed campaigns can create backlash. However, a negative ROI estimate could also indicate confounding that the model has not fully accounted for. Before acting, check whether the negative estimate is robust across model specifications.
Several legitimate reasons: (1) actual market changes (new competitors, economic shifts), (2) changes in media strategy or creative quality, (3) additional data improving estimation. However, if nothing changed strategically and results shift dramatically, this may indicate sensitivity to modeling choices. Ask for a sensitivity analysis comparing the two years under consistent specifications.
Platform metrics (Google Analytics, Meta) measure different things than an MMM. Platforms report attributed conversions (users who saw an ad and converted). MMM estimates incremental impact (conversions that would not have happened without the ad). These can differ significantly—platform metrics often overcount because they include conversions that would have happened anyway. Neither is “right”; they answer different questions.
MMMs are designed primarily for attribution and optimization, not forecasting. While the model can make forward predictions, those predictions carry additional uncertainty from unknown future conditions (economic changes, competitive actions). Use the posterior predictive distribution for forecasting, and treat the uncertainty intervals as minimums—real-world uncertainty is typically larger.
This is healthy and expected. Present the evidence clearly: show the data, the prior assumptions, the sensitivity analysis, and any experimental validation. If there is genuine domain knowledge that contradicts the model, it should be encoded as a prior in the next model run—not used to reject results post hoc. If disagreement persists, design an experiment to resolve it empirically.
More Resources
For the technical modeling workflow, see the Step-by-Step Modeling Guide. For understanding the risks of specification shopping, see For Business Stakeholders. For a deep dive into the mathematics, see the Technical Guide.