Cinelli Hazlett 2020 - Bounding Confounder Strength with Observed Covariates

Summary

To judge plausibility, an analyst often argues a confounder is “no more than $k$ times as strong as” an important observed covariate $X_j$. The paper turns such claims into tight bounds on $R_{D\sim Z|X}^2$ and $R_{Y\sim Z|D,X}^2$, in three variants (total $R^2$; partial given $X_{-j}$; partial given $X_{-j}$ and $D$). The key correction over informal practice: the bound on the outcome association contains a collider-bias factor ${\eta }^2\ge k_Y$.

Overview

Plausibility (“could such a confounder exist?”) is the hardest step. Direct judgment about $R^2$ values of an unnamed $Z$ is hard; relative judgments against a known $X_j$ are easier. For the bound to be valid it must identify (bound) the bias, which requires deriving new functions of the chosen measure of relative strength (Cinelli et al. 2019), not just reading off observed statistics.

Assume $Z\perp X$, or equivalently work with the part of $Z$ not linearly explained by $X$.

Main Content

Relative-strength parameters (Eq. 21; "type iii" bound used in the main text)

For an observed covariate $X_j$ with remaining covariates $X_{-j}$:

$$k_D:=\frac{R_{D\sim Z|X_{-j}}^2}{R_{D\sim X_j|X_{-j}}^2},k_Y:=\frac{R_{Y\sim Z|X_{-j},D}^2}{R_{Y\sim X_j|X_{-j},D}^2}.$$

$k_D\le 1$ is the claim that omitting $Z$ would hurt the treatment regression’s fit no more than omitting $X_j$.

Bounds on the sensitivity parameters (Eq. 22)

Write $R_x^2:=R_{D\sim X_j|X_{-j}}^2$ and $f_x^2:=R_x^2/(1-R_x^2)$. Then

$$R_{D\sim Z|X}^2=k_D{f}_{D\sim X_j|X_{-j}}^2(\text{feasible only if }{k}_D{f}_{D\sim X_j|X_{-j}}^2<1),$$ $$R_{Y\sim Z|D,X}^2\le {\eta }^2{f}_{Y\sim X_j|X_{-j},D}^2,\eta =\frac{\sqrt{k_Y}+\sqrt{r_{ZX}}}{\sqrt{1-r_{ZX}}},r_{ZX}:=R_{Z\sim X_j|X_{-j},D}^2=\frac{k_D{R}_x^4}{(1-k_D{R}_x^2)(1-R_x^2)}.$$

The bounds are tight: some confounder attains equality. Because ${\eta }^2\ge k_Y$, ignoring the second term (informal benchmarking) understates the confounder’s outcome association.

Verification

The expressions for $R_{D\sim Z|X}^2$ and $r_{ZX}$ were checked by simulation (independent $X_j,Z$, $D=a{X}_j+bZ+\epsilon$, $b^2=k_D{a}^2$, $n=2\times 10^6$): empirical $R_{Z\sim X_j|D}^2=0.1291$ vs formula $0.1283$; $R_{D\sim Z|X_j}^2=0.2883$ vs $0.2874$. The closed form for $\eta$ follows the Supp. B.2 chain (Eqs. 74-83) and matches the form implemented in the sensemakr R package (ovb_partial_r2_bound); the PDF text extraction garbles the printed algebra, so the notation $r_{ZX}$ is this note’s, not the paper’s.

Derivations (Supp. B)

B.1 Total $R^2$ comparison. If $R_{D\sim Z}^2=k_D{R}_{D\sim X_j}^2$ and $R_{Y\sim Z}^2=k_Y{R}_{Y\sim X_j}^2$ and $Z\perp X$, then $R_{D\sim Z+X}^2=R_{D\sim Z}^2+R_{D\sim X}^2$, so $R_{D\sim Z|X}^2=k_D{R}_{D\sim X_j}^2/(1-R_{D\sim X}^2)$ (Eqs. 51-54). The outcome analogue plus the recursive partial-correlation formula (Eq. 55) give $R_{Y\sim Z|X,D}$. The sign is chosen so the confounder reduces $|\hat{\tau }|$: if $\hat{\tau }>0$, $R_{Y\sim Z|D,X}$ and $R_{D\sim Z|X}$ share a sign; the worst case is $R_{Y\sim Z|X,D}=\frac{|R_{Y\sim Z|X}|-|R_{Y\sim D|X}{R}_{D\sim Z|X}|}{\sqrt{1-R_{Y\sim D|X}^2}\sqrt{1-R_{D\sim Z|X}^2}}$ (Eq. 56).

B.2 Partial $R^2$ comparison. From $R_{D\sim Z|X_{-j}}^2=k_D{R}_{D\sim X_j|X_{-j}}^2$ and $R_{Z\sim X_j|X_{-j}}=0$ (since $Z\perp X$): $R_{D\sim Z|X}=\sqrt{k_D}{f}_{D\sim X_j|X_{-j}}$, i.e. $R_{D\sim Z|X}^2=k_D{f}_{D\sim X_j|X_{-j}}^2$ (Eq. 63), subject to the feasibility constraint $k_D\le (1-R_{D\sim X}^2)/R_{D\sim X_j|X_{-j}}^2$ (Eq. 65). Example: if $X_j$ already explains 50% of residual treatment variance ($f^2=1$), no orthogonal $Z$ can be stronger. For the outcome, condition on $D$: $Z$ and $X_j$ become correlated given $D$ (collider), $R_{Z\sim X_j|X_{-j},D}^2=r_{ZX}$ above, and the recursive formula plus the triangle bound (Eqs. 79-83) yield $|R_{Y\sim Z|D,X}|\le \eta f_{Y\sim X_j|X_{-j},D}$ with equality attainable (Eq. 80 step).

Extension to groups. Replace $X_j$ by a set $X_{(1\dots j)}$ via the iterated recursive partial correlation (Eqs. 66-69). Different covariate groups may bound $D$ and $Y$ separately (e.g. $X_1$ the main driver of selection, $X_2$ the main driver of outcome).

Without conditioning on $D$ (Eqs. 70-72). Define $k_Y$ with $R_{Y\sim X_j|X_{-j}}^2$ (no $D$), then use Eq. 72 to convert to $R_{Y\sim Z|D,X}$.

When bounds can be trusted

Section 6.4 warns: “weak covariates can make the apparent bounds look better”. The method is valid only if the analyst can argue the benchmark covariates are among the strongest predictors of $D$ or $Y$.

Examples

Darfur: Female as the benchmark

Within a village, gender was likely the most important visible determinant of targeting (sexual violence), so the authors take $X_j=$Female. The bound for $k_D=k_Y=1$ is $R_{Y\sim Z|D,X}^2=12\%$, $R_{D\sim Z|X}^2=1\%$; $2\times$ gives $(24\%,2\%)$ and $3\times$ gives $(36\%,3\%)$ (points in Fig. 2). Estimates then fall to about 77%, 55% and 32% of the original.

import numpy as np
def ovb_bounds(r2dxj, r2yxj, kd=1.0, ky=1.0):
    """r2dxj = R2_{D~Xj|X-j}; r2yxj = R2_{Y~Xj|X-j,D}. Returns (R2_{D~Z|X}, R2_{Y~Z|D,X} bound)."""
    r2dz = kd*r2dxj/(1-r2dxj)
    assert r2dz < 1, "kd infeasible: data rule out a Z this strong"
    r2zx = kd*r2dxj**2/((1-kd*r2dxj)*(1-r2dxj))
    eta2 = ((np.sqrt(ky)+np.sqrt(r2zx))**2)/(1-r2zx)
    return r2dz, eta2*r2yxj/(1-r2yxj)
print(ovb_bounds(0.01, 0.10, 1, 1))   # illustrative inputs, not the Darfur Female values

Connections

• Plugs into the contour plots of Cinelli Hazlett 2020 - Darfur Application and Reporting.
• Why these bounds beat informal ones: Cinelli Hazlett 2020 - Informal Benchmarking Pitfalls and Related Methods.
• Benchmarking idea in Bayesian settings (calibrating the sensitivity prior): Franks 2019 - Calibrating Sensitivity Parameters.

See Also

Cinelli Hazlett 2020 - Multiple and Nonlinear Confounders, Cinelli Hazlett 2020 - Making Sense of Sensitivity - Overview