Synthetic Control, Done Right

Matthew Reda Causal inference

A brand launches in one big market, or goes dark in one region — and there is no second Chicago to compare against. Synthetic control stops searching for the twin and builds it: a convex weighted blend of untreated markets chosen to reproduce the treated market's own history. This post covers the estimator, when it is credible, the bias theory behind it, and the modern variants.

One Treated Market, No Twin

The standard playbook for measuring incrementality with observational panel data is difference-in-differences: compare the treated unit's change to a control group's change, leaning on parallel trends. That playbook breaks in exactly the setting marketing teams face most often. When a single aggregate unit is treated — one DMA gets the launch, one region gets the TV blackout — there is one treated observation per period, standard errors are undefined, and no individual control market is a plausible stand-in. Los Angeles is not a control for Chicago, and averaging all untreated markets equally imposes a comparison nobody would defend.

Abadie, Diamond, and Hainmueller's answer is to make the control-group construction itself the estimation problem: choose a weighted combination of untreated units — the donor pool — calibrated to track the treated unit's outcome trajectory before the intervention. Whatever mixture of markets reproduced Chicago's sales history for years is a defensible forecast of Chicago without the launch, and the post-launch gap between actual and synthetic Chicago is the treatment effect.

The Estimator and Its Weights

Index units \( j = 1, \ldots, J+1 \), with unit 1 treated and units \( 2, \ldots, J+1 \) forming the donor pool. Outcomes \( Y_{jt} \) are observed for \( T \) periods, of which \( T_0 \) precede the intervention.

Definition: Synthetic control estimator

The effect of treatment on the treated unit at time \( t > T_0 \) is \( \tau_{1t} = Y_{1t}^I - Y_{1t}^N \), the gap between the treated and untreated potential outcomes. The untreated outcome is estimated by a weighted average of donors,

$$ \hat{Y}_{1t}^{N} \;=\; \sum_{j=2}^{J+1} w_j^{*}\, Y_{jt}, \qquad \hat{\tau}_{1t} \;=\; Y_{1t} - \hat{Y}_{1t}^{N}, $$

with weights chosen to minimize the pre-intervention discrepancy between the treated unit's characteristics \( \mathbf{X}_1 \) (pre-period outcomes plus predictors) and the donor pool's \( \mathbf{X}_0 \):

$$ \mathbf{W}^{*} = \arg\min_{\mathbf{W}} \left( \sum_{h=1}^{k} v_h \Big( X_{h1} - \sum_{j=2}^{J+1} w_j X_{hj} \Big)^{2} \right)^{1/2} \quad \text{s.t.}\quad w_j \ge 0,\;\; \sum_{j} w_j = 1. $$

The \( v_h \) weight the relative importance of each predictor; Abadie (2021) recommends choosing them by cross-validation on held-out pre-intervention periods.

The constraint set is the whole method. Without it, the weight problem is just an upside-down regression — predict the treated series from the donor series — and with dozens of donors and a limited pre-period, ordinary least squares fits the pre-period essentially perfectly by giving every donor a weight, many of them negative. That perfect fit is overfitting: post-period, the unconstrained combination extrapolates wildly. Restricting to a convex combination — non-negative weights summing to one — confines the counterfactual to the convex hull of the donor pool. Interpolation, never extrapolation.

Convexity buys two more properties for free. First, sparsity: geometrically, the synthetic control is the projection of \( \mathbf{X}_1 \) onto the convex hull of the donors, and when the treated unit sits outside that hull the projection lands on a face with at most \( k \) vertices — at most \( k \) donors receive non-zero weight. Second, transparency: the counterfactual is a short, named recipe ("42% Austria, 22% United States, …") that reviewers can interrogate, unlike the dense, sign-flipping weights a regression implicitly assigns.

# illustrative
from scipy.optimize import fmin_slsqp

def sc_weights(X0, x1):
    """X0: (T0 x J) donor pre-period matrix; x1: (T0,) treated series."""
    J = X0.shape[1]
    loss = lambda w: np.sqrt(np.mean((x1 - X0 @ w) ** 2))
    return fmin_slsqp(loss, np.full(J, 1.0 / J),
                      f_eqcons=lambda w: np.sum(w) - 1.0,   # sum to 1
                      bounds=[(0.0, 1.0)] * J,              # non-negative
                      disp=False)

The canonical illustration is California's Proposition 99, a 1988 cigarette tax studied with state-level data from 1970 to 2000 and 38 donor states. The convex program puts non-zero weight on only five states, tracks California closely through 1988 without overfitting, then diverges: by 2000, actual California sits roughly 25 packs per capita per year below its synthetic counterpart, and the gap widens over time (Abadie, Diamond, & Hainmueller, 2010).

Building the counterfactual from a donor blend

The treated market's sales (green) against its synthetic control (dashed) — a convex blend of donor markets, here 45% A · 25% B · 30% C, fit to reproduce the pre-period trajectory. They track closely before the intervention (the pre-period RMSPE measures how closely); after it, the gap opens and grows. Drag the true effect and watch the post-period divergence — the gap is the estimated lift.

15
Estimated lift (avg post gap)—
Final-period gap—
Pre-period fit (RMSPE)—

Because the weights are non-negative and sum to one, the synthetic control interpolates inside the donor hull — and any failure to match shows up as a visible pre-period gap, not hidden behind extrapolation.

When the Method Applies

Synthetic control's transparency cuts both ways: a bad application is visible in a way a bad regression is not. Abadie (2021) catalogs the conditions a credible application must satisfy, and they translate directly into marketing terms.

On the data side, the method needs aggregate outcomes and predictors for all units, an extended pre-intervention window (more on why below), and enough post-period for slow-building effects to show. The more volatile the outcome, the longer the pre-window must be.

⚠️ When not to use synthetic control

Skip the method when no convex combination of donors can approximate the treated unit; when the usable donor pool has fewer than about five units (permutation inference becomes meaningless); when the expected effect is small relative to volatility and the post-window is short; or when anticipation contaminates the pre-period and backdating is implausible. Mechanical application produces confident, misleading numbers.

Why Pre-Period Fit Controls Bias

The formal justification rests on a linear factor model for the untreated outcome:

$$ Y_{jt}^{N} \;=\; \delta_t \;+\; \boldsymbol{\theta}_t \mathbf{Z}_j \;+\; \boldsymbol{\lambda}_t \boldsymbol{\mu}_j \;+\; \varepsilon_{jt}, $$

where \( \delta_t \) is a common trend, \( \mathbf{Z}_j \) are observed covariates with time-varying coefficients, \( \boldsymbol{\mu}_j \) are unobserved unit-specific factor loadings, \( \boldsymbol{\lambda}_t \) are time-varying common factors, and \( \varepsilon_{jt} \) is idiosyncratic noise. Difference-in-differences is the special case where \( \boldsymbol{\lambda}_t \) is constant — unobserved confounders shift every period identically, which is precisely parallel trends. The factor model relaxes it: each market can have its own sensitivity to macro conditions, category trends, or competitive dynamics that drift over time.

The central result of Abadie, Diamond, and Hainmueller (2010) is a bound on the bias of \( \hat{\tau}_{1t} \) under this model. Provided the synthetic control closely reproduces the treated unit's pre-period trajectory and characteristics, the bias is inversely proportional to \( T_0 \), increasing in the donor-pool size \( J \), and controlled by the fit discrepancy \( \mathbf{X}_1 - \mathbf{X}_0\mathbf{W}^{*} \). Three rules fall out. A long pre-treatment window is genuinely valuable, not decorative. An imperfect pre-period fit is a stop sign, not a caveat. And a larger donor pool is not automatically better — extra donors add flexibility to overfit noise, and scope for interpolating across markets that resemble the treated unit on average but not individually.

Deep diveA sketch of the bias bound

Under the factor model, the post-period gap between the treated unit and any fixed-weight donor combination (the trend \( \delta_t \) cancels because weights sum to one) is

$$ \hat{\tau}_{1t} - \tau_{1t} \;=\; \boldsymbol{\theta}_t \Big( \mathbf{Z}_1 - \textstyle\sum_j w_j \mathbf{Z}_j \Big) \;+\; \boldsymbol{\lambda}_t \Big( \boldsymbol{\mu}_1 - \textstyle\sum_j w_j \boldsymbol{\mu}_j \Big) \;+\; \Big( \varepsilon_{1t} - \textstyle\sum_j w_j \varepsilon_{jt} \Big). $$

The covariate term is observable and small when the fit is good. The dangerous term is the second: the mismatch in unobserved loadings, amplified by whatever the common factors do post-period. A close pre-period fit disciplines exactly this term: matching the treated outcome in every one of \( T_0 \) pre-periods means matching \( \boldsymbol{\lambda}_t \boldsymbol{\mu} \)-combinations at \( T_0 \) different factor values, and if the factors vary enough, the only way to satisfy all those constraints is to genuinely match the loadings, \( \sum_j w_j \boldsymbol{\mu}_j \approx \boldsymbol{\mu}_1 \).

The catch is noise: the weights could instead achieve fit by chasing the realized \( \varepsilon_{jt} \). The bound quantifies how much of the apparent fit can be spurious. At a fixed donor count, each additional pre-period adds a constraint without adding flexibility, so the expected spurious component shrinks — the bias bound scales inversely with \( T_0 \). Growing \( J \) at fixed \( T_0 \) does the opposite, which is why "throw every market into the donor pool" is counterproductive. Convexity is what keeps the residual bias visible: unable to extrapolate, the method reports any failure to match as an explicit discrepancy \( \mathbf{X}_1 - \mathbf{X}_0\mathbf{W}^{*} \), where an unconstrained regression hides the same problem behind extrapolation.

Inference Without Standard Errors

With one treated unit and a donor pool of ten to fifty aggregate entities, asymptotic standard errors are unavailable and arguably meaningless — states and DMAs are not draws from a superpopulation. The canonical alternative is permutation (placebo) inference: pretend, in turn, that each donor was treated, fit its synthetic control from the remaining donors, and ask whether the actually-treated unit's result is extreme within this placebo distribution. The randomness modeled is treatment assignment, not sampling — a design-based test in Fisher's tradition.

The raw post-period gap is a poor test statistic: a donor whose synthetic control fit poorly pre-period shows a large post-period gap by construction. The recommended statistic normalizes by pre-period fit.

Definition: RMSPE ratio

For unit \( j \), let \( R_j(t_1, t_2) \) be the root mean squared prediction error of its synthetic control over periods \( t_1, \ldots, t_2 \). The test statistic is the ratio of post-intervention to pre-intervention RMSPE,

$$ r_j \;=\; \frac{R_j(T_0 + 1,\, T)}{R_j(1,\, T_0)}, $$

and the permutation p-value is the fraction of units — treated unit included — whose ratio is at least as large as the treated unit's:

$$ p \;=\; \frac{1}{J+1} \sum_{j=1}^{J+1} \mathbf{1}\!\left\{ r_j \ge r_1 \right\}. $$

In-space placebo inference: the gap between each unit and its own synthetic control. Every donor reassigned as “treated” (grey) hovers near zero throughout, while the actually-treated market (green) is flat pre-intervention and diverges sharply after — an RMSPE ratio more extreme than any placebo, which is the permutation p-value.

Donors whose pre-period fit is hopeless are typically dropped from the placebo distribution before computing the p-value (a common screen keeps units whose pre-period RMSPE is within a small multiple of the treated unit's), and one-sided variants buy power when the direction of the effect is known in advance. In the Proposition 99 study, California's year-2000 effect of roughly −25 packs per capita was more extreme than every one of the 34 surviving placebo runs — a permutation p-value near 0.03.

Two diagnostics belong in every application. Backdating (an in-time placebo) reruns the analysis with the intervention date moved earlier: a credible synthetic control should show near-zero "effects" in periods when nothing happened, as the West Germany reunification analysis does when backdated a decade. Leave-one-out refits the synthetic control dropping each positively-weighted donor in turn: if the conclusion flips when a single donor exits, that donor deserves scrutiny — it may have had its own idiosyncratic shock. Interval estimates exist too: test inversion over hypothesized effect sizes (Firpo & Possebom, 2018) and prediction intervals (Cattaneo, Feng, & Titiunik, 2021).

💡 Preregister the design

Because weights, donor pool, and predictors are all functions of pre-intervention data, every design decision can be locked before the first post-launch data point arrives: fix the donor pool and predictors, fit and circulate the weights, then let the outcome data come in. This removes the specification-search degree of freedom that makes after-the-fact observational readouts easy to distrust — the closest observational work gets to a randomized trial's pre-analysis plan.

Modern Extensions

The canonical method assumes one treated unit, a decent hull, and complete data. Each assumption now has a well-developed relaxation.

Augmented and bias-corrected synthetic control

When the hull condition is strained — the fit discrepancy is non-trivial but not disqualifying — a regression adjustment can absorb the residual imbalance: fit an outcome model \( \hat{\mu}_{0t}(\cdot) \) on the donors and apply the synthetic control to residuals,

$$ \hat{\tau}_{1t} \;=\; \big( Y_{1t} - \hat{\mu}_{0t}(\mathbf{X}_1) \big) \;-\; \sum_{j} w_j^{*} \big( Y_{jt} - \hat{\mu}_{0t}(\mathbf{X}_j) \big). $$

This is the augmented synthetic control of Ben-Michael, Feller, and Rothstein (2021) — a doubly-robust move: the weights handle what interpolation can reach, the outcome model corrects what it cannot, and the correction matters most exactly when the treated unit drifts outside the hull.

Penalized weights and relaxed constraints

With several treated units, fitting one synthetic control per unit raises a uniqueness problem: many weight vectors can achieve the same fit. The penalized estimator of Abadie and L'Hour (2021) adds a term charging each donor for its distance from the treated unit, \( \lambda \sum_j w_j \| \mathbf{X}_1 - \mathbf{X}_j \|^2 \), restoring uniqueness and sparsity — interpolating between nearby donors rather than distant ones. At the other end, Doudchenko and Imbens (2016) drop convexity entirely: an elastic-net regression of the treated series on donor series, with an intercept to absorb level differences. That handles treated units outside the hull at the price of extrapolation and less interpretable weights — the trade the canonical method deliberately refuses.

Generalized synthetic control: estimate the factor model directly

Xu (2017) takes the linear factor model that justifies synthetic control and simply estimates it, as an interactive fixed effects model on the control units:

$$ Y_{it} \;=\; \delta_{it} D_{it} \;+\; x_{it}'\boldsymbol{\beta} \;+\; \boldsymbol{\lambda}_i' \mathbf{f}_t \;+\; \varepsilon_{it}. $$

The procedure has three steps: estimate the factors \( \mathbf{f}_t \) and coefficients on control data; project each treated unit's pre-treatment outcomes onto the estimated factor space to recover its loadings \( \hat{\boldsymbol{\lambda}}_i \); impute the counterfactual \( \hat{Y}_{it}(0) = x_{it}'\hat{\boldsymbol{\beta}} + \hat{\boldsymbol{\lambda}}_i'\hat{\mathbf{f}}_t \). Cross-validation on held-out pre-periods picks the number of factors, and a parametric bootstrap yields standard errors and confidence intervals — frequentist uncertainty the canonical method cannot offer. GSC handles many treated units with staggered timing in one run, and constant loadings recover difference-in-differences as a special case. Xu's voter-turnout application is a caution for anyone trusting two-way fixed effects by default: where DiD estimated an effect under one percentage point with visibly failing pre-trends, GSC with two factors estimated roughly five points with a near-perfect pre-period fit. His diagnostics carry over to marketing panels — plot the imputed counterfactuals, and check that treated units' factor loadings sit inside the hull of the control loadings, since loading-space extrapolation is the method's own bias source. Relatedly, matrix-completion estimators (Athey et al., 2021) treat the untreated-outcome matrix as low-rank and impute the missing entries directly, scaling to panels with many units, many periods, and missing data.

MethodTreated unitsKey assumptionInference
Canonical SC (ADH 2010)OneConvex hull, long pre-periodPermutation (RMSPE ratio)
Augmented SC (2021)One or fewOutcome model corrects hull failurePermutation / jackknife variants
Penalized SC (2021)MultipleLocal interpolation (penalty \( \lambda \))Permutation
Elastic-net SC (2016)OneRegularized regression; allows extrapolationCross-validation based
GSC / IFE (Xu 2017)Multiple, staggeredFactor model on controls; large \( N_{co}, T_0 \)Parametric bootstrap SEs
Matrix completion (2021)MultipleLow-rank outcome matrixBootstrap

One sibling deserves mention because it competes for the same job: Bayesian structural time-series, the engine behind CausalImpact, forecasts the treated series from donor series inside a state-space model and reads the effect off a posterior rather than a permutation distribution. When the donor pool is thin but the treated series has rich temporal structure, it is often the better tool — the comparison is drawn out in the companion post on CausalImpact and BSTS.

A Launch-Market Playbook

Pulling the threads together for a single-market launch or blackout:

  1. Before launch, assemble the donor pool: unexposed markets, no media bleed, no competing interventions, structurally similar to the treated market. Gather as long a pre-period as the data allow; deseasonalize if the outcome is volatile.
  2. Lock the design: predictors, donor pool, V-matrix procedure, and the weights themselves, all fixed before outcome data arrive. Inspect the fit discrepancy — if no convex combination of donors resembles the treated market, choose a different method.
  3. After the readout window, run the placebo battery: in-space permutation with the RMSPE ratio, backdating, leave-one-out. Report the effect alongside its placebo rank, not just the gap chart.
  4. Feed the result forward. A synthetic-control lift estimate is exactly the kind of experimental evidence that calibrates a media mix model's channel priors — one well-measured market can discipline a parameter the national time series never could.

Takeaways

  • Synthetic control replaces the search for a twin market with a construction: a convex, sparse, named weighted average of donors that reproduces the treated market's pre-period trajectory.
  • Convexity is the point — interpolation only, visible fit discrepancies, weights that can be preregistered before outcomes exist.
  • Bias is controlled by long pre-periods and close fit; imperfect pre-period fit is a stop sign, and bigger donor pools are not automatically better.
  • Inference is by placebo permutation on the RMSPE ratio, backed by backdating and leave-one-out — not standard errors.
  • When the assumptions strain, reach for the extensions: augmented SC for hull failures, penalized SC for multiple treated units, GSC for staggered treatment with bootstrap intervals.

🧰 This method ships in the framework

Synthetic control is a first-class named method in the experiment toolkit (synthetic_control in the planning.methods registry): demeaned convex donor weights, the RMSPE-standardized placebo-permutation p-value described above, and power/MDE calibrated by the A/A·A/B simulation harness on your own history. It appears alongside TBR, GBR, and matched-market DiD in the Design Studio's method picker and the methodology leaderboard.

References