Cinelli Hazlett 2020 - Traditional OVB and Sensitivity Contours

Summary

The classical OVB identity says that omitting a confounder $Z$ biases the treatment coefficient by $\hat{\gamma }\hat{\delta }$: the confounder’s “impact” on the outcome times its “imbalance” across treatment. Plotting the adjusted estimate ${\hat{\tau }}_{res}-\hat{\gamma }\hat{\delta }$ over a grid of $(\hat{\delta },\hat{\gamma })$ gives a bivariate sensitivity contour plot, but the raw-coefficient parameterisation is unit-dependent and does not extend to many confounders or standard errors, which motivates the partial-$R^2$ version.

Overview

The paper starts from two regressions. The investigator would like the full model but cannot observe $Z$, so runs the restricted one:

$$Y=\hat{\tau }D+X\hat{\beta }+\hat{\gamma }Z+{\hat{\epsilon }}_{full}(3)$$ $$Y={\hat{\tau }}_{res}D+X{\hat{\beta }}_{res}+{\hat{\epsilon }}_{res}(4)$$

Here $Y$ is the outcome, $D$ the treatment (binary or continuous), $X$ the $n\times p$ matrix of observed pre-treatment covariates (including the constant), $Z$ a single unobserved covariate. The target is $\hat{\tau }$ (from the full model) and the observed quantity is ${\hat{\tau }}_{res}$.

Main Content

Omitted variable bias (sample version)

Let $A^{\perp X}$ denote the residual of $A$ after linear projection on $X$ (Frisch-Waugh-Lovell). Define the sample bias $\widehat{\text{bias}}:={\hat{\tau }}_{res}-\hat{\tau }$ and the imbalance $\hat{\delta }:=\frac{\text{cov}(D^{\perp X},Z^{\perp X})}{\text{var}(D^{\perp X})}$, the coefficient on $D$ in the reverse regression $Z=\hat{\delta }D+X\hat{\psi }+{\hat{\epsilon }}_Z$. Then

$${\hat{\tau }}_{res}=\hat{\tau }+\hat{\gamma }\hat{\delta },\widehat{\text{bias}}=\hat{\gamma }\hat{\delta }.(5)\text{-}(6)$$

Derivation (Eq. 5). By FWL, ${\hat{\tau }}_{res}=\text{cov}(D^{\perp X},Y^{\perp X})/\text{var}(D^{\perp X})$. Substituting $Y^{\perp X}=\hat{\tau }{D}^{\perp X}+\hat{\gamma }{Z}^{\perp X}+{\hat{\epsilon }}_{full}$, and using that ${\hat{\epsilon }}_{full}$ is orthogonal to $D$ and $X$, gives ${\hat{\tau }}_{res}=\hat{\tau }+\hat{\gamma }\text{cov}(D^{\perp X},Z^{\perp X})/\text{var}(D^{\perp X})$.

Interpretation: impact times imbalance

• $\hat{\gamma }$ (“impact”): difference in expected outcome per unit of $Z$ at the same $D,X$.
• $\hat{\delta }$ (“imbalance”): coefficient of the reverse regression of $Z$ on $D$ and $X$, not of $D$ on $Z$. It is the conditional imbalance of $Z$ across treatment, the same object checked in covariate-balance tests.
• Whatever the true functional form of $E[Z|D,X]$ (i.e. whatever the assignment mechanism), only its linear imbalance enters the bias. No model of treatment assignment is needed.
• Footnote 7: $\hat{\gamma }$ is associational. It is a causal “impact” only under additional structural assumptions; $Z$ is only required to close back-door paths.

Sensitivity contour plot (Fig. 1)

Each hypothesised pair $(\hat{\delta },\hat{\gamma })$ maps to the adjusted estimate $\hat{\tau }={\hat{\tau }}_{res}-\hat{\gamma }\hat{\delta }$. The paper parameterises so the bias hurts the preferred hypothesis (reduces $|\hat{\tau }|$); the opposite direction is handled symmetrically.

Darfur: the Center confounder

Suppose a critic argues that people at the village center were more often harmed and also have different peace attitudes. With unadjusted ${\hat{\tau }}_{res}=0.097$ and an imbalance $\hat{\delta }=0.25$ (injured respondents 25 percentage points more likely to live in the center), the effect is zero only if $\hat{\gamma }\ge 0.097/0.25\approx 0.39$; the paper reads from Fig. 1 that the impact “would have to be over 0.40” on a 0 to 1 PeaceIndex. Given relatively homogeneous villages, the authors argue this is hard to believe.

The move from “perfect identification vs a critic’s list of confounders” to “how much confounding would matter” is the main value of the exercise.

Shortcomings of raw-coefficient OVB (motivation for Sec. 4)

1. Units. Center is binary so impact and imbalance are easy to read. For a latent trait like Political Attitudes, doubling the scale halves the needed impact and doubles the needed imbalance. Standardising does not help if the target is in original units.
2. Multiple confounders. How do Political Attitudes and Wealth, acting jointly and non-linearly, enter the plot?
3. Benchmarking. How can all unnamed confounders be benchmarked against an observed covariate like Female?
4. Inference. The plot says nothing about standard errors, $t$-values or intervals.

Examples

# Adjusted estimate over an (imbalance, impact) grid: Darfur-style numbers
import numpy as np
tau_res = 0.097
delta = np.linspace(0, 0.4, 5)       # imbalance grid
gamma = np.linspace(0, 0.4, 5)       # impact grid
adj = tau_res - np.outer(gamma, delta)   # rows gamma, cols delta
print(np.round(adj, 3))
# zero contour: gamma * delta = 0.097
print("gamma needed at delta=0.25:", tau_res / 0.25)

Connections

• Mechanically identical to the omitted-variable formula in Omitted Variables Bias; see also Logic of Regression Adjustment.
• Bayesian version: put priors on $(\gamma ,\delta )$ and push through (5); see Cinelli Hazlett 2020 - Planted Confounders and Bayesian Sketch.
• Next: Cinelli Hazlett 2020 - Partial R2 Parameterization and Bias Formulas.

See Also

Sensitivity Analysis in Observational Studies, Potential Outcomes Framework, Cinelli Hazlett 2020 - Making Sense of Sensitivity - Overview