Frame first. Then model.
propensity_lift estimates brand lift from exposed-versus-control survey data. This guide covers the package, and the decisions that have to be made before you run it.
propensity_lift estimates brand lift from exposed-versus-control survey data. This guide covers the package, and the decisions that have to be made before you run it.
If you build response or targeting models, the mechanics here are familiar: a classifier, a score, some weights. What changes is the job. A targeting model succeeds by predicting exposure well. A propensity analysis succeeds when the comparison it sets up is fair, and a model that predicts exposure too well is a sign that no fair comparison exists.
The package reports balance, overlap and confidence intervals. All three can look excellent while the answer is wrong, because each of these is settled outside the data.
Every cause of exposure that also moves the outcome is in the model. This is unconfoundedness, and it cannot be tested.
No unmeasured confoundingFor every kind of exposed respondent there are controls like them. This is overlap, and the package can show you where it fails.
Overlap and positivityOne well-defined exposure, with no spillover from one respondent to another. This is consistency, and it is decided when you code the flag.
A well-defined exposureThese studies had far richer data than a survey and still went wrong, because the variables that drove exposure were not in the data. The estimator was rarely the weak point.
Lewis, Rao and Reiley (2011). A randomised test measured a 5.4% effect on brand searches. Regression with every available control said 872%.
Gordon et al. (2019). In seven of the 14 Facebook experiments with a checkout outcome, the observational estimates were off by more than a factor of three.
Gordon, Moakler and Zettelmeyer (2023). Over 663 experiments, the median upper-funnel lift from stratified propensity score matching, against the median lift the experiments measured.
Give it a respondent-level table and a config. It fits the propensity model, checks overlap, applies every adjustment method you asked for, and reports lift for each KPI with diagnostics next to every number.
from propensity_lift import AnalysisConfig, load_table, run_analysis
df = load_table("survey.xlsx")
cfg = AnalysisConfig(
dataset_id="survey",
treatment_column="group", exposed_values=["Exposed"], control_values=["Control"],
outcome_columns=["aided_awareness", "favorability", "consideration"],
model_features=["age", "social_media_hours_day", "prior_brand_usage"],
methods=["nn_matching", "ipw", "overlap_weights", "aipw"],
estimand="ATT",
)
result = run_analysis(df, cfg)
Nearest-neighbour matching, inverse probability weighting, overlap weights, stratification and doubly robust AIPW run from one config. When they disagree, that is information.
How each method worksCommon support, the overlap coefficient and feature-by-feature gaps, reported before any lift number.
Reading overlapStandardised mean differences for every covariate, with the same denominator on both sides so the two are comparable.
Balance diagnosticsThe bootstrap redoes the matching or weighting on every resample, so the interval reflects the adjustment step and not only the final difference in means.
UncertaintyStart with the question: what you are estimating, for whom, and which variables may go in the model.
Frame the questionAn end-to-end run on the bundled sample survey, with every output explained as it appears.
WalkthroughEvery function, config option and result field, generated from the source.
API reference