Evaluator's Guide

At a Glance

License	Proprietary (copyright Matthew Reda, all rights reserved). Versions up to and including 1.5.0 were released under Apache-2.0 and remain available on PyPI under that license; later versions are closed-source. Licensing inquiries: mattreda@mattreda.pro.
Version	1.2.0 (2026-07-25) — current release; 1.0.0 was the first stable release (2026-07-24); in development since December 2025; first published to PyPI as 0.1.0 in June 2026. Strict SemVer from 1.0.0; Experimental-tier modules may still change in minor releases (see the stability tiers).
Packaging	On PyPI: `pip install mmm-framework`. Pin an exact version for production use (`mmm-framework==1.5.0`). The base install is the lean modeling core; the agent stack is the `[agents]` extra, and the FastAPI web app ships as the separate `mmm-framework-server` package.
Maintainer	Matthew Reda, solo maintainer. This is a real risk; see Team & sustainability for what mitigates it.
Support	Email mattreda@mattreda.pro. No SLA. No commercial support contract is currently offered.
Requirements	Python 3.12+. Core stack: PyMC 6.0+ (PyTensor 3, ArviZ 1.x), NumPyro, FastAPI backend, React frontend.
Test suite	3,751 passing tests (3,587 fast + 164 slow incl. real MCMC fits, measured 2026-07-06 on the PyMC 6 stack).
Governance	Single-maintainer project. No foundation, no steering committee—continuity rests on the documented methodology, the seeded notebook evidence, and the recorded scorecards rather than on a license backstop.

Maturity & API Stability

Different modules have matured at different rates, and the changelog page maintains the authoritative tier list. As of 1.0 the summary is:

Stable (frozen at 1.0) — the core modeling surface: BayesianMMM, results classes, the fluent builders, Pydantic configs, the adstock/saturation/seasonality/trend transforms, the MFF data loader, the declarative estimands registry, serialization (the persisted model format), and the REST API (mmm-framework-server) with its org/tenant auth.
Beta — the active surface: reporting, analysis, the React web UI, the AI agent workspace, the measurement-loop planning engine, the pre-fit EDA module, and the diagnostics/validation suites.
Experimental — subject to change: the extension models (NestedMMM, MultivariateMMM, CombinedMMM), the continuous-learning loop, the Model Garden / Atelier, the non-MMM model families, the DAG model builder, and the Excel config workflow.

From 1.0.0 the project commits to strict SemVer: breaking changes to Stable APIs require a major version, while modules flagged Experimental may still change in minor releases (with deprecation warnings where feasible).

1.0 shipped 2026-07-24

The criterion set out on this page — stabilizing the Beta-tier APIs — was met, and 1.0.0 shipped on 2026-07-24. (PyPI publication, the other original criterion, happened at 0.1.0.) Pinning an exact release for production use remains good hygiene: pip install mmm-framework==1.5.0.

What's Measured vs. What's Claimed

The marketing for most MMM tools is assertion. This project's policy is that every quantitative claim on the docs site traces to a seeded, re-runnable notebook—so any number you read can be re-derived by re-baking the notebook, not taken on trust. The four proof assets an evaluator should weigh:

Pressure-testing scorecard. The framework is fit to 16 synthetic worlds with known causal ground truth, and every estimate is graded against that truth—including the 8 worlds where it fails silently. The recorded scorecard, with failures, is public: pressure-testing.
Forecast accuracy backtest. Rolling-origin out-of-sample accuracy is measured, not asserted: forecasting workflow — backtesting.
Runtime benchmark. A production-size national fit (156 weeks × 7 channels, NumPyro, 4 chains × 500 draws) measures ~15 s wall-clock on an Apple M3 laptop (PyMC 6), including compilation. Source: nbs/validation/runtime_benchmark.ipynb; full scaling table in the technical guide's Computational Scaling section. Absolute seconds are machine-specific; treat the ratios as the transferable part.
Real-data test. The framework is run on the Lydia Pinkham advertising series (1907–1960), the most-studied real dataset in advertising econometrics, and graded against decades of published estimates: nbs/validation/pinkham_real_data.ipynb.

The methodology itself is documented in prose—the technical guide and the mathematics series—rather than living only in code, so it can be audited without reading the source.

Team & Sustainability

This framework is built and maintained by one person: Matthew Reda. That is the single largest risk in adopting it, and you should weigh it as such. If the maintainer is unavailable, there is no team to pick up the work. We are not going to spin that.

What genuinely mitigates the bus factor:

A test suite of 2,000+ passing tests that lets a successor maintainer modify the code with a safety net.
Pre-registered, seeded notebook evidence. Every measured claim on this site comes from a notebook that can be re-baked—the evidence base does not depend on the maintainer's memory.
A pressure-testing harness with a recorded scorecard that is re-recorded whenever framework defaults change, so regressions in methodology are visible, not silent.
Methodology stated in documentation, not hidden in code. The technical guide and math series describe what the model does and why, so the design intent survives independently of any individual.

If your procurement process requires a vendor with an organization behind it, this project does not currently meet that bar. If it requires software your team can audit against a recorded evidence base, the items above are designed for exactly that.

Security & Deployment

The framework is self-hosted: a Python library, a FastAPI agent backend that runs model fits in-process (no Redis or worker required), and a React frontend that you run on your own infrastructure. (Redis is an optional shared rate-limit backend that defaults to in-memory.) There is no hosted SaaS offering, which means no client data leaves your environment by default.

The AI agent workspace introduces an LLM data flow that deserves separate scrutiny—which provider sees what, sandboxing of agent-executed code, and the configuration options (including fully local models via LM Studio). That is covered on the dedicated security page rather than duplicated here.

Data Requirements & Runtime

Input data uses the MFF (Master Flat File) long format, loaded and validated by MFFLoader. The practical floor and the recommended shape of the data—granularity, history length, channel coverage—are documented on the data requirements page. Short version: 104+ weeks of weekly data is the comfortable starting point.

Runtime is measured, not estimated: the reference production-size national fit completes in ~15 s on a laptop (see Computational Scaling), geo panels scale sub-linearly per row, and the extension models stay in the same runtime ballpark as the core model. Model iteration speed is not a procurement risk here.

How to Evaluate This in 30 (or 120) Minutes

30 minutes — reading only

Read the pressure-testing scorecard first, including the failure column—a tool that publishes where it breaks is showing you its actual epistemics. Then the forecast backtest for out-of-sample honesty, and skim Aurora 05 to see the full workflow end to end.

120 minutes — hands on

Do the 30-minute pass, then pip install mmm-framework and follow the quickstart: fit a first model on synthetic data with known ground truth, and check the estimates against the answer key yourself.

Evaluating it as the engine of a consulting practice? See the 10–12 week engagement timeline and the full set of printable consultant artifacts (checklists, pre-registration memo, executive summary template) the framework generates.

Known Limitations

⚠️ Read before depending

Stable 1.0 on PyPI. Pin an exact version (mmm-framework==1.5.0) for anything production-facing and read the release notes before upgrading; SemVer applies from 1.0.0.
Solo maintainer. No SLA, no commercial support, real bus-factor risk (mitigations above).
Experimental-module churn. Modules flagged Experimental may change in minor releases; the Stable tier is frozen under SemVer.
No SaaS offering. You operate the stack (the mmm-framework-server FastAPI agent API + React frontend) yourself.
Open methodological findings. The pressure-testing page records methodological limits openly—e.g. regional ROI inversion under performance-chased budgets when media effectiveness is pooled across geos (mitigated by the opt-in per-geo vary_media_by_geo hierarchy or a per-geography refit). (Two former headline findings were since resolved: the legacy blend adstock underperforming the parametric path—fixed by making parametric adstock the default—and the holdout-validation saturation fallback for Hill-fit models.) See known strengths & weaknesses.

None of these are hidden in fine print elsewhere—this list is the fine print, on purpose. If any item is disqualifying for your context, better to know now.