Marketing-mix models on mmm-framework¶

From v0.5, all marketing-mix models (MMM) on the platform run on mmm-framework (PyPI mmm-framework==1.5.0, Apache-2.0). mmm-framework is a Bayesian MMM framework that uses PyMC. mapi-plugin-mmm puts it behind the platform contract as four built-in models:

Model	In plain words	Engine
`mmm.bayesian`	Measures how much each marketing channel adds to your sales, in total or per market. It also shows what will occur if you move budget between channels.	mmm-framework `BayesianMMM`: NUTS (nutpie, PyMC or NumPyro), or fast approximations (MAP, ADVI, Pathfinder)
`mmm.frequentist`	Gives a first estimate in seconds. Its ranges come from samples of the data, not from the uncertainty of a full Bayesian model.	mmm-framework's ridge estimator with bootstrap intervals
`mmm.nested` v0.12	Measures marketing effects that go through an intermediate result, for example visits or awareness.	mmm-framework `NestedMMM` (`mmm_extensions`): NUTS (nutpie or PyMC) or ADVI
`mmm.multivariate` v0.12	Measures marketing effects on two or more results at the same time, and the effects between the results.	mmm-framework `MultivariateMMM` (`mmm_extensions`): NUTS (nutpie or PyMC) or ADVI

Each model has an About card in plain language. The card tells what the model is for, when not to use it, what data it needs and what you get from it. Help me choose in the web app recommends these models for the goals "explain my marketing" and "plan my budget" (mapi choose -a goal=plan_budget, see Choosing a model). mmm-framework is the only marketing-mix engine on the platform. The platform itself optimises budgets (below).

What the data looks like¶

mmm-framework does the scaling, the carry-over and the conversion back to your units (owns: [scaling, carryover, inverse_units]). Thus, the pipeline only converts currencies, adds the data up to weeks and adds controls. The spec validation refuses a scaling step or a lag step on the KPI or on the spend (CAPABILITY_UNSUPPORTED). Thus, the platform does not scale a value two times.

The table gives the canonical names. Bind the names of your dataset to them in data.bind:

Name	Dims	Role
`time`	-	the period dim (`extend`: forecasts continue after the data)
`channel`	-	one label for each media channel (`fixed`)
`geo`, `product`	-	the panel dims in the list `hierarchy.pool_over` (none for one national series)
`y`	`(*panel, time)`	the KPI
`spend`	`(*panel, channel, time)`	the spend for each channel (`future_required`: a forecast needs it)
any other numeric variable over time	`(*panel?, time, ...)`	a control (or put them in the list `controls`). Each label of an extra dim, for example a Fourier mode, becomes one feature.

apiVersion: mapi/v1
kind: ModelSpec
metadata: {name: eu-geo-mmm, project: growth}
data:
  dataset: dataset://growth/eu_media_daily@latest
  side_tables:
    fx:       {ref: dataset://growth/fx_daily@latest, score_policy: supplied}
    holidays: {ref: dataset://shared/holidays_eu@v3, score_policy: pinned}
  bind:
    dims: {geo: market, channel: channel, time: date}
    vars: {y: revenue_local, spend: spend_local}
  split: {kind: time_holdout, dim: time, last: 8, after_step: weekly}
pipeline:
  - {id: to_eur, use: core.fx_convert@^1,
     with: {vars: [y, spend], table: fx, table_along: date, currency_dim: geo,
            currencies: {UK: GBP}, default_currency: EUR}}
  - {id: weekly, use: core.resample@^1, with: {dim: time, freq: W-SUN, how: sum, full_periods_only: true}}
  - {id: hols,   use: core.holidays@^1, with: {table: holidays, table_along: date, along: time}}
model:
  use: mmm.bayesian@^1
  settings:
    hierarchy: {pool_over: [geo]}
    new_levels: hyperprior
    adstock: {kind: geometric, l_max: 6}
    saturation: {kind: hill}
    trend: {kind: linear}
    seasonality: {yearly_order: 3}
  fit: {mode: evaluate_then_refit, engine: nutpie, tune: 1000, draws: 1000, chains: 4, seed: 7}
outputs:
  derived: [contributions, response_curves, marginal_cost]
  plots: [core.ppc_fan, core.convergence, mmm.decomposition, mmm.response_curves]
evaluate: {metrics: [elpd_loo, coverage_90, mape]}

The plugin includes a smaller version of this spec that you can run: mapi_plugin_mmm.examples.E33_SPEC_YAML. It has a simulator (e33_dataset) that gives known true contributions. NATIONAL_SPEC_YAML with national_dataset() is the smallest national example.

MFF data¶

The native format of mmm-framework is the Master Flat File (MFF). It has one row for each Period, Geography, Product, Campaign, Outlet, Creative and VariableName. The value is in VariableValue.

Upload the file with the mff ingest preset, or as a Guided upload. Each variable becomes its own data variable. The key columns have the same meaning as in the MFF of mmm-framework:

Geography and Product become dims when they have more than one level.
Some variables have only a total: a blank, National or Total label. An example is national TV next to regional sales. The platform divides such a variable across the levels in the same way as MFFLoader. Each geography gets an equal part. Each product gets a part that agrees with its share of the KPI. MffOptions.allocation changes this.
A variable cannot have both rows for each level and total rows.
The platform adds Campaign, Outlet and Creative up to the dims of the model, unless MffOptions.split keeps them.
The Guided geo template puts the media variables that you select into spend(channel).

The plugin gives the panel to mmm-framework in the same layout that its own MFFLoader makes. The observations are in period order first, then geography, then product. Tests compare an MFF file that the platform ingests with the same file that mmm_framework.load_mff loads. They make sure that the two give the same panel, with the national media rows and the campaign rows.

How the spec maps onto mmm-framework¶

Spec	mmm-framework
`y`, `spend`, controls	`PanelDataset` (`y`, `X_media`, `X_controls`) with `MFFConfig` (`KPIConfig`, `MediaChannelConfig` for each channel, `ControlVariableConfig` for each control feature)
`hierarchy.pool_over: [geo, product]`	the `Geography`/`Product` dimensions of the KPI; `HierarchicalConfig.pool_across_geo/product` (a baseline for each level, partially pooled)
`hierarchy.media_by_geo`	`HierarchicalConfig.vary_media_by_geo` (channel effects for each geo, partially pooled)
`adstock.kind`, `l_max`, `normalize`	`AdstockConfig` (`geometric`, `delayed`, `weibull`, `none`) on all channels
`saturation.kind`	`SaturationConfig` (`logistic`, `hill`, `michaelis_menten`, `tanh`, `root`, `none`)
`channels.<c>.adstock/saturation`	the configs of that channel
`trend.kind`	`TrendConfig` (`none`, `linear`, `piecewise`, `spline`, `gp`)
`seasonality.yearly_order` ...	`SeasonalityConfig.yearly/monthly/weekly` (Fourier pairs)
`media_prior: roi`	`ModelConfig.media_prior_mode="roi"`: the return on spend sets the parameters of the channel effects
`priors.roi.<c>: {mu, sigma}`	`MediaChannelConfig.roi_prior_mu/roi_prior_sigma`
`priors.intercept: {mu, sigma}`	`ModelConfig.intercept_prior_mu/sigma`
`likelihood: student_t`	`LikelihoodConfig(family=student_t)`
`likelihood: poisson`, `negative_binomial`, `beta`	a platform subclass of `BayesianMMM` that replaces `_build_likelihood` (see Counts and shares)
`time_varying.channels`, `innovation_sigma`	`MediaChannelConfig(time_varying=True, tvp_innovation_sigma=...)` for each channel in the list
`fit.engine`	`nutpie`/`pymc`/`numpyro`: NUTS with that backend. `map`, `advi`, `fullrank_advi`, `pathfinder`, `laplace`, `smc`: `BayesianMMM.fit(method=...)`
`fit.tune`, `draws`, `chains`, `target_accept`, `seed`	the same `fit()` arguments

The plugin reads frequency (W, D or M) from the time labels. If the labels are integers, set frequency. When you score, new_levels sets what occurs for new panel levels:

pooled: the overall level and the average channel effects.
hyperprior: for each draw, a level from the fitted spread. Thus, its range is correctly wider.
zero: no channel effects.
error.

What is stored, and how scoring works¶

A fit keeps these items. It never keeps them as a pickle:

The parameters of mmm-framework (posterior).
The sampler statistics.
A thinned prior of the free parameters (for the prior/posterior overlap on the scorecard).
The observed KPI.
A fit_data group with the KPI, the spend and the control features at the model stage.
A lift_tests group when the fit used lift tests v0.12: the value and the standard error of each test, and the estimate of the model for each draw. The mmm.lift_tests chart reads it.

mmm.nested and mmm.multivariate keep the same groups. Their thinned prior comes from a prior predictive sample of the graph.

When you score, the plugin makes the BayesianMMM again from fit_data. The same data give the same scaling, Fourier features and trend basis. Then the plugin calculates the fitted model again in NumPy for each posterior draw. This calculation includes the intercept, trend, seasonality, panel levels, the adstock and saturation of each channel, each control and the observation noise. It is the same forward pass as the PosteriorForecaster of mmm-framework. Tests compare it with the forecaster and with the deterministics of the graph, to floating-point precision.

Carry-over. The plugin writes the scored periods over the fitted history. Thus, a TV burst of last month has an effect on the sales of next month. A four-week window gives the same scores as the same weeks of the full series. Periods after the fit window must follow it with no gap.
Trends after the data. The plugin extends a linear trend. Piecewise, spline and GP trends stay at their last fitted level (the forecast policy of mmm-framework). The forecast tells you this. Their intervals do not become wider with the horizon.
Outputs. y_hat(*panel, time) and contributions(component, *panel, time). There is one component for each channel, baseline (intercept and panel levels), trend, seasonality and one for each control. They add up to y_hat. A predictive request adds noise from a seeded generator. Thus, the scenario and the baseline of a counterfactual use the same noise.

Priors in business terms¶

The return on spend sets the parameters of the channel effects. With media_prior: roi (the default), each channel samples roi ~ LogNormal(mu, sigma). This is the KPI that the channel returns for each unit of spend over the fit window. mmm-framework calculates the coefficient from it. Thus, you give a prior in the units that a business owner uses:

settings:
  priors:
    roi:
      tv:     {mu: 0.69, sigma: 0.4}    # median ROI e^0.69 = 2.0, 90% within about [1.0, 3.9]
      search: {mu: 1.1, sigma: 0.3}
    roi_default: {mu: 0.0, sigma: 1.0}  # channels not listed: median break-even
    intercept: {mu: 0.2, sigma: 0.3}    # the baseline, in KPI standard deviations

The platform changes business priors into these priors in closed form. Business priors come from Guided task 3, from calibration.business_priors or from a translate_priors run:

An ROI range [low, high] with credibility p becomes the log-normal that has this range as its central p interval.
A range for the contribution share is an ROI range, multiplied by the KPI total divided by the spend of the channel (share = roi x spend / KPI).
The platform reads a range that starts at 0 as [high / 20, high], and gives an info issue.
The baseline share (the result with no media) sets the intercept prior. The platform calculates it by bisection against the prior of mmm-framework for the trend, seasonality, levels and controls.

The preview samples the prior predictive of the translated model (the graph of mmm-framework). It shows the ROI, the share, the baseline share and a KPI fan in business units. It also shows the prior mass in each range that you gave (coverage). This value is near the credibility that you gave.

Lift tests¶

calibration.lift_tests become experiment likelihoods in mmm-framework (BayesianMMM.add_experiment_calibration). Each is a likelihood term on the estimand that the model gives. It updates the effect of the channel, its saturation curve and its carry-over together.

Lift test	mmm-framework measurement
extra spend (`spend_delta > 0`)	`MROAS`, `spend_lift_pct = 100 x spend_delta / window spend`, value `kpi_delta / spend_delta`
a holdout that removes all (or almost all) of the spend of the channel (`-spend_delta >= 95%` of it)	`CONTRIBUTION` of `-kpi_delta`
a partial cut (less than 95% of the spend)	skipped. The reason is in `learned.calibration.lift_tests.skipped`. This test measures the slope below the current spend, `(f(S) - f((1-c)S)) / cS`. `MROAS` is the slope above it (`spend_lift_pct > 0` only). When returns decrease, the two slopes are different (by `e^(c lam)` for `1 - exp(-lam x)`). Thus, the mapping would bias the curve.

geo limits the measurement to that geo. The plugin skips a test after the data. It also skips a test that the fit window cuts (a holdout fit that ends in the test). It puts the skipped tests in a list.

From v0.13, a geo-lift test that causal.synthetic_control measured can become a lift test. The workflow node model.lift_test makes the entry from the impact group of the causal model version. The workflow template geo_lift_mmm does the test, the lift test and the calibrated fit in one run. The report of the Guided template geo_lift shows the same entry to copy. See Feed the result to the marketing mix model and From a geo-lift test to a lift test for the MMM.

Derived quantities¶

Ask for them in outputs.derived. The plugin calculates them at fit time over the first outputs.ui_draws draws. It keeps them in the bundle:

contributions: channel_contribution(sample, channel, *panel, time), baseline, trend, seasonality and one variable for each control.
response_curves: contribution(sample, channel, multiplier[, geo]). This is the total contribution over the fit window, with the spend of the channel multiplied by the multiplier (0 to 2.5 x by default). It also gives the spend on each grid point, the current spend, the largest spend in one period and the baseline. The what-if sliders of Guided mode interpolate these values for each draw. The plausibility checks of the scorecard read them.
marginal_cost: the extra spend for each extra unit of KPI at the current spend, and its inverse.
adstock v0.12: weight(sample, channel, lag). This is the part of the total effect of one unit of spend that comes lag periods later. The weights of each draw add up to 1. mean_lag(sample, channel) is the mean delay. Every MMM has it.
effectiveness v0.12 (mmm.bayesian, mmm.frequentist): effectiveness(sample, channel, time). This is the KPI for each unit of spend at the average spend of the fit window, with the coefficient of period time. It is constant for a channel that is not time-varying. time_varying(channel) tells which channels change.
mediation (mmm.nested) and cross_effects (mmm.multivariate): refer to the sections of these models below.

Charts v0.12¶

Put the recipes in outputs.plots. Each recipe reads the derived quantities or bundle groups in the table. A recipe that does not find its data gives a clear error and the other plots continue.

Recipe	Reads	Shows
`mmm.roi`	`response_curves`, optional `marginal_cost`	The return on spend of each channel with a range, and the marginal return (mROAS), against break-even
`mmm.adstock`	`adstock`	How the effect of spend decays over the next periods, with a range
`mmm.waterfall`	`contributions`	The contribution of each component to the total result
`mmm.spend_effect`	`response_curves`	The share of spend against the share of the effect of each channel
`mmm.lift_tests`	the `lift_tests` group	Each lift test with its error bar, next to the estimate of the model
`mmm.effectiveness`	`effectiveness`	The effect of one unit of spend over time (time-varying channels)
`mmm.mediation`	`mediation`	The direct and indirect effect of each channel
`mmm.cross_effects`	`cross_effects`	The effects between the results
`core.energy`	`sample_stats.energy`	The energy plot of the sampler and the BFMI of each chain
`core.prior_posterior`	`prior`, `posterior`	The prior and the posterior of each parameter, with their overlap

mmm.waterfall and mmm.decomposition have the setting outcome. With an outcome dim (mmm.multivariate), they show the primary outcome unless you select a different one. Plots reference gives the settings of each recipe.

Budget optimisation¶

POST /v1/models/{model_id}/versions/{version}:optimize asks how to divide a budget across channels (and, optionally, geos). An optimize run in the sandbox makes the response curves of the fitted model for each posterior draw. It gives them to the planning module of mmm-framework:

optimize_budget: greedy marginal allocation, or a constrained multi-start solver for risk objectives, groups and money bounds.
budget_frontier.

curl -s -X POST "$API/v1/models/$MODEL/versions/3:optimize" \
  -H "Authorization: Bearer $MAPI_TOKEN" -H "X-Mapi-Project: growth" \
  -H "Prefer: wait=60" -H "Content-Type: application/json" -d '{
    "total_budget": 1000000,
    "channels": {"tv": {"min_multiplier": 0.5, "max_multiplier": 1.5},
                 "search": {"min_spend": 50000}},
    "groups": [{"name": "digital", "channels": ["search", "social"], "min_share": 0.4}],
    "objective": "p10",
    "frontier": {"n_points": 10}
  }'

Request	mmm-framework
`total_budget` / `budget_change_pct`	the budget (default: the current total)
`mode: fixed` / `free`	spend the budget / give each channel money up to break-even (`value_per_kpi` converts KPI to money)
`channels.<c>.min_multiplier/max_multiplier`	`bounds` (multiples of the current spend)
`channels.<c>.min_spend/max_spend`	`abs_bounds` (money; they override multiples)
`channels.<c>.locked`	the multiplier stays at 1
`groups`	portfolio constraints (shares and/or money)
`objective`	`mean`, `p<q>` (a cautious percentile) or `cvar<a>` (the mean of the worst draws)
`by_geo`	allocate for each (geo, channel) arm together
`horizon`	the periods of the plan (the curves are scaled again from the fit window at its average spend)
`max_draws`, `n_steps`, `seed`	sampling controls

The answer is 200 if the run finished in the Prefer: wait time. If not, the answer is 202. Then poll GET /v1/optimizations/{run_id}. The answer contains these items:

The allocation for each channel, with its expected contribution and marginal return. It also tells how stable the recommendation is across draws.
The expected KPI of the plan and of the current plan.
The uplift and the probability that it is positive.
The expected regret if you use one plan.
The shadow price of the next unit of money.
The efficient frontier.
Notes. For example, a note tells you when a recommendation is more than the spend that the model has seen.

The run also makes the mmm.budget_plans plot. If the data cannot meet the bounds, the run fails with OPTIMIZE_INFEASIBLE.

End to end with the client¶

These steps go from a fitted version to a plan that you can defend, in Python. The equivalent CLI commands follow.

Do a check of the model first. A plan is only as good as its response curves. Run the governance policy for marketing-mix models. It does null tests against shuffled sales and false noise channels. It also does a check of the effect signs and the holdout accuracy. Read why each check passed:

validation = api.models.validate(model_id, version, note="Q4 refresh", wait=True)
print(validation.status.value)  # validated, awaiting_approval or checks_failed
for r in validation.results:
    print(r.check, r.status.value, r.message_plain, r.suggested_action or "")
if validation.status.value == "checks_failed":
    validation = api.validations.resubmit(validation.id, note="refitted on clean data")

Look at the curves. The response_curves section of the Model Explorer shows the contribution of each channel when its spend changes, for each posterior draw. A mark shows the current spend. decomposition shows the part that the model gives to each channel over time:
```
explored = api.models.explore(model_id, version, sections=["response_curves", "decomposition"])
curves = api.explorations.view(explored, "response_curves")
```

Ask for the plan. api.models.optimize sends Prefer: wait. It sends the request again with the same Idempotency-Key until the run in the sandbox is done. Thus, a slow plan never starts two times:

plan = api.models.optimize(  # mmm-framework's planner on the posterior; waits for the plan
    model_id,
    version,
    budget_change_pct=0,  # reallocate today's total
    channels={"tv": {"min_multiplier": 0.5, "max_multiplier": 1.5}},
    objective="p10",  # plan for a cautious outcome, not the average one
    frontier={"n_points": 8},
)
assert plan.result is not None
for a in plan.result.allocation:
    print(f"{a.channel:10} {a.current_spend:>12,.0f} -> {a.optimal_spend:>12,.0f}")
uplift = plan.result.uplift
print("uplift", uplift.mean if uplift else None, "P(better)", plan.result.prob_positive_uplift)

Test the largest change. Score the weekly spend of the plan as a forecast. Then design a lift test for the channel that the plan changes most. Put its result into the calibration block of the model (see Lift tests). The next plan uses it.

You can do the same steps from a terminal:

mapi validate model model://growth/geo-mmm@v7
mapi explore model model://growth/geo-mmm@v7 -s response_curves -s decomposition --out views/
mapi optimize model://growth/geo-mmm@v7 --change-pct 0 \
  --channel 'tv={"min_multiplier": 0.5, "max_multiplier": 1.5}' \
  --objective p10 --frontier 8

This output shows the plan for the two-channel model of the demo (eu-geo-mmm). The model is the fit from the Python client guide, with very small sampler settings and half a year of data. The output is from a laptop:

 optimisation                      run_01m53hqpb8690ja1nkrc7b8j12
 objective                         downside P10 (a low quantile — risk-averse)
 budget                            639,176 EUR (now 639,176)
 expected revenue_local            1.656e+07 (now 1.658e+07)
 uplift                            -1.431e+04
 chance it beats the current plan  48%
 draws                             200
Plan
 channel ┃ geo ┃ now     ┃ recommended ┃ change ┃ next unit returns ┃ note
━━━━━━━━━╇━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━
 tv      │     │ 370,845 │ 488,240     │ +32%   │ 2.92              │
 search  │     │ 268,331 │ 150,936     │ -44%   │ 2.94              │
What more (or less) budget buys
 budget    ┃ expected revenue_local ┃ next unit returns
━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━
 255,670   │ 1.504e+07              │ 4.15
 547,865   │ 1.622e+07              │ 3.9
 840,060   │ 1.725e+07              │ 3.43
 986,157   │ 1.766e+07              │ 2.78
 1,132,254 │ 1.784e+07              │ 1.24
 1,278,352 │ 1.79e+07               │ 0.41
 …
note: Non-concave response curve(s) detected for tv, search: the greedy allocator's exactness
precondition fails, so this run used the multi-start constrained solver instead. Treat the
optimum as the best point found, not a proven global one.

Read the output as the planning module intends. The plan protects the cautious result (the 10th percentile). Thus, its expected revenue is a little less than the current revenue. The plan is better than the current plan in only 48% of the draws. On this model, money that moves from search to TV gives insurance, not growth. The next euro returns about the same (2.9) in the two channels. Thus, the current split is already almost efficient.

The frontier is the more useful answer. Up to about 1 million EUR, each extra euro returns 2.8 or more. Above 1.13 million, the return decreases to 1.2 and then to 0.4. The notes (on standard error) tell which solver ran and why. With --objective mean, the same request maximises the expected revenue.

A workflow can do all these steps on a schedule. The mmm_governed gallery template fits mmm.bayesian on a geo panel. It runs the policy with submit: true and keeps the evidence pack (mapi workflows init mmm_governed).

Effects that change over time v0.12¶

The effect of a channel can change, for example when an advertisement gets old or when a competitor starts to spend. Put the channel in time_varying.channels:

model:
  use: mmm.bayesian@^1
  settings:
    adstock: {kind: geometric, l_max: 8}
    saturation: {kind: hill}
    time_varying:
      channels: [tv]
      innovation_sigma: 0.15   # how fast the effect can change from one period to the next
      forecast: random_walk    # random_walk (default), hold_last or mean

The coefficient of a time-varying channel is exp(level + sigma x cumsum(z)), with the cumulative sum centred over the fit window. This is a random walk on the log coefficient. beta_<channel> is its mean over time. The platform keeps the path of each draw and uses it when you score. In the fit window, each period uses its own coefficient.

After the fit window, time_varying.forecast sets the coefficient:

random_walk: the walk continues from the last period, with the fitted step size. The range becomes wider with the horizon. A scenario and its baseline use the same path.
hold_last: the coefficient of the last fitted period.
mean: the mean coefficient of the fit window.

A forecast tells you the policy in its notes. These rules also apply:

A time-varying channel cannot have an ROI prior (priors.roi). The validation refuses the spec. A business prior on its ROI also gives an error.
Lift tests on a time-varying channel are allowed. The test uses the coefficients of its own periods.
With hierarchy.media_by_geo, a time-varying channel has one national path. The fit tells you this.
Response curves and budget plans use the average effect over the fit window. The decision objective and steady_state use the coefficient of the last fitted period.

The effectiveness derived quantity and the mmm.effectiveness chart show the path of each channel.

Counts and shares v0.12¶

The default likelihood is normal (or student_t). For a count, such as orders or sign-ups, use poisson or negative_binomial. For a share between 0 and 1, such as a market share, use beta:

model:
  use: mmm.bayesian@^1
  settings:
    likelihood: negative_binomial
    media_effect_sigma: 0.5     # half-normal prior of each channel effect on the log scale
    adstock: {kind: geometric, l_max: 8}
    saturation: {kind: hill}

Likelihood	Data	Link	Extra parameter
`poisson`	integers, 0 or more	log	none
`negative_binomial`	integers, 0 or more	log	`nb_alpha ~ Gamma(2, 0.1)` (dispersion)
`beta`	values between 0 and 1 (not 0 and not 1)	logit	`beta_kappa ~ Gamma(2, 0.05)` (precision)

The fit does a check of the data. If a value does not agree with the likelihood, the fit stops with SCHEMA_INVALID at /data. With these likelihoods, the KPI is not standardised. Thus, these rules apply:

The channel effects use coefficient priors (HalfNormal(media_effect_sigma)), not ROI priors. priors.roi and priors.roi_default give a validation error. The default intercept prior is centred on log(mean y) or logit(mean y).
Business priors and lift tests give CALIBRATION_UNSUPPORTED.
The terms add up on the link scale, not in KPI units. The platform splits the expected KPI into parts that add up to y_hat. The baseline is the KPI with no media. Each channel gets a part of the rest, in proportion to its term on the link scale. The trend, the seasonality and the controls are part of the baseline. The score tells you this.
Response curves and the return on spend are removal-based: the KPI with the actual spend minus the KPI with no spend on the channel. A predictive score draws counts (or shares) from the likelihood with seeded random numbers. A scenario and its baseline use the same random numbers.

The nested model (`mmm.nested`) v0.12¶

mmm.nested measures effects that go through an intermediate measure (a mediator). For example, TV ads increase brand awareness, and awareness increases sales. Each mediator has its own equation. The channels that you select act on it. The mediator then acts on the KPI. The channels can also act on the KPI directly.

Data. One national series. The model has no geo or product dims and takes no control variables.

Name	Dims	Role
`time`	-	the period dim (`extend`)
`channel`	-	one label for each media channel (`fixed`)
`mediator`	-	one label for each intermediate measure (`fixed`)
`y`	`(time)`	the KPI
`spend`	`(channel, time)`	the spend for each channel (`future_required`)
`mediators`	`(mediator, time)`	the measured values. Missing values are allowed: a survey can have values in some periods only.

model:
  use: mmm.nested@^1
  settings:
    trend: {kind: linear}
    seasonality: {yearly_order: 2}
    likelihood: normal           # or student_t
    mediators:
      - name: awareness
        channels: [tv, social]   # the channels that move it (default: all)
        direct_effect: true      # the channels also act on the KPI directly
        observation_sigma: 0.1   # noise of the survey, in mediator standard deviations
      - name: visits
        outcome_sign: positive   # the effect of visits on the KPI is not negative
  fit: {mode: evaluate_then_refit, engine: nutpie, tune: 1000, draws: 1000, chains: 4, seed: 7}
outputs:
  derived: [contributions, response_curves, marginal_cost, adstock, mediation]
  plots: [core.ppc_fan, core.convergence, mmm.decomposition, mmm.mediation, mmm.roi]

With latent: true, the platform does not read the column of that mediator (it can be all empty). Then only the product of the two paths is identified. The chart shows the paths in KPI units, which are identified. With an empty mediators list, each label of the mediator dim gets the defaults.

How it maps onto mmm-framework. The engine fixes the media transforms: a normalised geometric adstock with 8 lags and the saturation 1 - exp(-lambda x). The media is scaled by its maximum, the KPI is standardised, and each measured mediator is standardised with its own mean and standard deviation (kept in learned).

Outputs.

y_hat(time).
contributions(component, time): one component for each channel (its total effect: direct plus indirect), baseline, trend and seasonality. They add up to y_hat.
mediator_hat(mediator, time): the fitted mediators, in their units (latent mediators in standard units).

Derived quantity mediation. direct(sample, channel), indirect(sample, channel, mediator) and total(sample, channel) are KPI totals over the fit window at the actual spend. share_mediated(sample, channel) is the part of the total that goes through the mediators. mmm.mediation draws them.

Lift tests calibrate the total effect of a channel, in the same way as for mmm.bayesian. Business priors are not available (CALIBRATION_UNSUPPORTED).

The multivariate model (`mmm.multivariate`) v0.12¶

mmm.multivariate measures the effect of one marketing budget on two or more results, for example the sales of a premium and a basic range. Each result (outcome) has its own equation. The noise of the results is correlated. Cross-effects let the sales of one product move the sales of a different product.

Data. One national series of each outcome. The model takes no control variables.

Name	Dims	Role
`time`	-	the period dim (`extend`)
`channel`	-	one label for each media channel (`fixed`)
`outcome`	-	one label for each result (`fixed`)
`y`	`(outcome, time)`	the KPIs
`spend`	`(channel, time)`	the spend for each channel (`future_required`)

model:
  use: mmm.multivariate@^1
  settings:
    trend: {kind: linear}
    seasonality: {yearly_order: 2}
    outcomes:
      - {name: premium, weight: 3.0}   # one unit of premium is worth 3 units in the plan
      - {name: basic, weight: 1.0}
    primary_outcome: premium
    cross_effects:
      - {source: premium, target: basic, kind: halo}
    lkj_eta: 2.0
    share_adstock: true
  fit: {mode: evaluate_then_refit, engine: nutpie, tune: 1000, draws: 1000, chains: 4, seed: 7}
outputs:
  derived: [contributions, response_curves, marginal_cost, adstock, cross_effects]
  plots: [core.ppc_fan, core.convergence, mmm.decomposition, mmm.cross_effects, mmm.roi]

Cross-effect `kind`	Prior	Sign
`halo`	half-normal (`prior_sigma`)	positive: the source helps the target
`cannibalization`	negative half-normal	negative: the source takes sales from the target
`unconstrained`	normal	either

Cross-effects are associations

Do not read a cross-effect as proof that one product takes sales from a different product. In the engine, a cross-effect acts on the observed sales of the source. Thus, it is confounded with the correlation of the noise. Use a test to measure a cause.

Scores and plans use the reduced form. In the engine, the source term uses the observed source outcome. A forecast has no observed outcome. Thus, the platform solves Y = (I - P)^-1 c for each draw and period, with P[k, s] = psi[s, k]. A channel that moves outcome A then also moves outcome B through the cross-effect. If the cross-effects of a draw make a loop with no solution (|det(I - P)| < 1e-6), the score fails with CAPABILITY_UNSUPPORTED.

Outputs. y_hat(outcome, time) and contributions(component, outcome, time): the channels, baseline, trend and seasonality, through the reduced form. They add up to y_hat for each outcome. The attribute primary_outcome names the primary outcome.

One value for planning. Response curves, marginal costs, the ROI metrics and budget plans use the weighted total sum_k weight_k x outcome_k. Give weights that convert each outcome to the same units (for example, the margin of one unit). response_curves also has contribution_by_outcome and baseline_by_outcome. When only one outcome has a weight that is more than 0, the budget plan and the response curves give the name of that outcome as the KPI. A note on the plan gives its weight.

Derived quantity cross_effects. psi(sample, pair) with the coords source, target and kind of each pair ("premium→basic"), in target units for each unit of source. mmm.cross_effects draws them.

This model has no lift-test calibration, no business priors and no promotion modulation.

The frequentist model¶

mmm.frequentist uses the same data and settings (but no priors, likelihood or GP trend). mmm-framework finds the adstock and saturation with an out-of-sample (rolling-origin) criterion, and then fixes them. It solves the model, which is now linear, by ridge regression with channel effects that are not negative. Then it solves the model again on block-bootstrap samples. These replicates are the "draws" of all outputs. Thus, the ranges are confidence intervals.

This model has no convergence diagnostics, business priors or lift tests. The seed sets its results, and it fits in seconds.

model:
  use: mmm.frequentist@^1
  settings:
    hierarchy: {pool_over: [geo]}
    adstock: {kind: geometric, l_max: 6}
    saturation: {kind: logistic}
    bootstrap: 200
    search_budget: 200
  fit: {mode: evaluate_then_refit, seed: 7}

estimator: constrained (a convex program) needs the optional cvxpy dependency of mmm-framework. The images of the platform do not install it. Thus, the fit fails with CAPABILITY_UNSUPPORTED.

Run tracking v0.11¶

Each fit logs params and metrics to its run (v0.12). The params tell the version of mmm-framework, the number of channels, periods and cells, and the link. The metrics give the return on spend of each channel with a range (roi, roi_p05, roi_p95), the marginal return (mroi), the share of each channel and the baseline share. Thus you can search, sort and compare MMM runs by the return of a channel:

mapi runs ls --filter "metric.roi{channel=tv} > 2" --column "metric.roi{channel=tv}"

The NUTS fits of mmm.bayesian, mmm.nested and mmm.multivariate also send the progress of each chain and the step metrics. See What an MMM fit logs, Live progress and cancel and Compare runs and use experiments.

Current limits¶

The platform does not give these mmm-framework features: the multiplicative (log-log) specification, price and promotion levers, events and interactions between channels.
mmm.nested and mmm.multivariate use national data only and take no control variables. Their media transforms are fixed (geometric adstock with 8 lags, exponential saturation).
mmm.multivariate has no lift-test calibration and no promotion modulation.
The likelihoods binomial, beta-binomial and lognormal are not available. They need a trials variable or a multiplicative path.
A budget plan of a count model (log link) adds the increments of the channels. Under a log link, the channels do not add up exactly in KPI units. The removal-based response curves are the correct view of each channel.
The approximate fits (ADVI) of mmm.nested and mmm.multivariate do not send their loss while they fit.
Fits run on the CPU. Only the NumPyro backend uses JAX.

Trust, but verify¶

The four models pass the SDK contract kit (mapi_sdk.testing.check_model). The kit tests these items:

Bundle round trips.
Shuffled markets and subsets of markets.
New levels.
Typed errors for gaps and for periods before the data.
A holdout that the plugin scores through the same path as a forecast.
Additive contributions.
The plots.

The conformance corpus fits these specs from start to end: e33 (a geo panel with a GBP market, side tables and a weekly plan), guided_mmm (business priors and a lift test) and mmm_frequentist. A recovery test fits simulated data with known contributions. It makes sure that the split of the contributions across channels is correct.