Rosenman 2020 - Variance-Weighted Shrinkage and Practical Estimation

Summary

Letting the shrinkage factor scale with each stratum’s RCT variance gives ${\kappa }_2={\hat{\tau }}_r-{\lambda }_2^{URE}{\Sigma }_r({\hat{\tau }}_r-{\hat{\tau }}_o)$: noisy RCT strata lean on the observational data, precise ones do not. The same procedure gives dominance (Lemma 3) and asymptotic optimality (Theorem 3). In practice, variances come from within-arm sample variances (biased upward under treatment-effect heterogeneity), and the observational estimate should use stabilised IPW to cut bias at little variance cost.

Overview

${\kappa }_2$ is the heteroscedastic analogue of ${\kappa }_1$: component-wise rather than common shrinkage. It tends to over- or under-shrink some strata when variance estimates are off, which is why the simulations find common shrinkage (${\kappa }_1$ family) usually wins.

Main Content

${\kappa }_2$ family (Sec. 4.3)

Structure $\kappa (\lambda {\Sigma }_r,{\hat{\tau }}_r,{\hat{\tau }}_o)={\hat{\tau }}_r-\lambda {\Sigma }_r({\hat{\tau }}_r-{\hat{\tau }}_o)$. Minimising the URE:

$${\lambda }_2^{URE}=\frac{\text{Tr}({\Sigma }_r^{2}D)}{({\hat{\tau }}_o-{\hat{\tau }}_r{)}^T{\Sigma }_r^{2}D({\hat{\tau }}_o-{\hat{\tau }}_r)},{\kappa }_2={\hat{\tau }}_r-\frac{\text{Tr}({\Sigma }_r^{2}D){\Sigma }_r}{({\hat{\tau }}_o-{\hat{\tau }}_r{)}^T{\Sigma }_r^{2}D({\hat{\tau }}_o-{\hat{\tau }}_r)}({\hat{\tau }}_r-{\hat{\tau }}_o),$$ $${\kappa }_{2+}={\hat{\tau }}_o+\left[I_K-\frac{\text{Tr}({\Sigma }_r^{2}D){\Sigma }_r}{({\hat{\tau }}_o-{\hat{\tau }}_r{)}^T{\Sigma }_r^{2}D({\hat{\tau }}_o-{\hat{\tau }}_r)+}\right]({\hat{\tau }}_r-{\hat{\tau }}_o),$$

and corrected ${\kappa }_{2+}^{\ast }$ with scale $a_2^{\ast }=1-\frac{2({\hat{\tau }}_o-{\hat{\tau }}_r{)}^T{\Sigma }_r^4{D}^2({\hat{\tau }}_o-{\hat{\tau }}_r)}{({\hat{\tau }}_o-{\hat{\tau }}_r{)}^T{\Sigma }_r^{2}D({\hat{\tau }}_o-{\hat{\tau }}_r)}\cdot \frac{1}{\text{Tr}({\Sigma }_{r}D)}$. (Exact positive-part placement as printed; the PDF extraction of the ”+” subscript is ambiguous.)

Lemma 3 (dominance)

If $4{\max }_k{d}_k^2{\sigma }_{rk}^4\le {\sum }_k{d}_k^2{\sigma }_{rk}^4$ then ${\kappa }_2$ dominates ${\hat{\tau }}_r$. Proof by the same argument as Lemma 1.

Theorem 3 ( ${\kappa }_2$ asymptotic risk)

Under mild conditions, as $K\to \infty$ ${\kappa }_2$ has the lowest risk among all estimators with a variance-weighted shrinkage factor. Proof (Appendix 9) parallels Theorem 2 with stronger moment conditions (Lemma 6): $\mathrm{limsup}\frac{1}{K}{\sum }_k{d}_k^2{\sigma }_{rk}^6{\xi }_k^2<\infty$, $\mathrm{limsup}\frac{1}{K}{\sum }_k{d}_k^2{\sigma }_{rk}^6{\sigma }_{ok}^2<\infty$, $\mathrm{limsup}\frac{1}{K}{\sum }_k{d}_k^2{\sigma }_{rk}^8<\infty$.

Variance estimation (Sec. 4.4.1)

${\Sigma }_r$ is unknown; following Green et al. (2005) replace it by

$${\hat{\sigma }}_{rk}^2=\frac{1}{n_{rkt}}\sum _{i\in R_k}{W}_i(Y_i-{\bar{Y}}_{rkt}{)}^2+\frac{1}{n_{rkc}}\sum _{i\in R_k}(1-W_i)(Y_i-{\bar{Y}}_{rkc}{)}^2$$

(as printed in the paper; the Neyman variance estimator has $1/\{n(n-1)\}$ factors for each arm, i.e. $s_t^2/n_{rkt}+s_c^2/n_{rkc}$, so treat the printed normalisation as shorthand), with ${\bar{Y}}_{rkt},{\bar{Y}}_{rkc}$ the arm means. It is unbiased under Assumption 2 (constant effect within stratum). If effects are heterogeneous within strata it overestimates the variance (Imbens & Rubin 2015), which for ${\kappa }_{1+}$ yields too much shrinkage toward ${\hat{\tau }}_o$ (the factor is linear in $\text{Tr}({\Sigma }_{r}D)$) and for ${\kappa }_{2+}$ may over-shrink some and under-shrink others. Mitigations: smaller strata (so Assumption 2 is likelier), or a “menu” of estimators computed under various assumed potential-outcome correlations (correlation 1 = Assumption 2; Imbens & Rubin Chapter 6).

Propensity adjustment (Sec. 4.4.2)

Because the observational study is typically much larger, adjusting by the estimated propensity score is good practice (bias drops, variance rise is modest). If strata are $K$ treatments fit a propensity model per arm; if subgroups, one model for all of $O$.

Stabilised IPW (SIPW) estimator of stratum $k$

$${\hat{\tau }}_{ok}={\left(\sum _{i\in O_k}\frac{W_i}{{\hat{p}}_i}\right)}^{-1}\sum _{i\in O_k}\frac{W_i{Y}_i}{{\hat{p}}_i}-{\left(\sum _{i\in O_k}\frac{1-W_i}{1-{\hat{p}}_i}\right)}^{-1}\sum _{i\in O_k}\frac{(1-W_i)Y_i}{1-{\hat{p}}_i}$$

(Horvitz-Thompson with normalised weights). It admits the Zhao et al. (2019) sensitivity analysis used in Rosenman 2020 - Sensitivity Analysis and Implied Gamma.

Examples

import numpy as np
def sipw_ate(y, w, p_hat):
    """Stabilised (self-normalised) IPW estimate of the treatment effect within one stratum."""
    w1 = w / p_hat; w0 = (1-w) / (1-p_hat)
    return (w1*y).sum()/w1.sum() - (w0*y).sum()/w0.sum()
 
def kappa2_plus(tau_r, tau_o, sigma2_r, d=None):
    K = len(tau_r); d = np.ones(K) if d is None else d; D = d/K
    num = (sigma2_r**2 * D).sum()
    den = max(((tau_o-tau_r)**2 * sigma2_r**2 * D).sum(), 1e-12)
    lam = num/den
    shrink = np.clip(lam*sigma2_r, 0, 1)             # component-wise weight toward tau_o
    return tau_r - shrink*(tau_r - tau_o)

Connections

• Marketing: the “strata” can be campaigns, each with a holdout lift test (noisy) and a propensity-based estimate (precise but biased); noisier holdouts borrow more from the observational estimate, as in Rosenman 2020 - Hierarchical Bias Model Sketch for Few Holdouts.
• Propensity machinery: Propensity Score and the Balancing Property, Propensity Score in Bayesian CI.

See Also

Rosenman 2020 - Combining Observational and Experimental Datasets - Overview