Second Brain
Search
Search
Dark mode
Light mode
Explorer
Tag: topic/online-experimentation
14 items with this tag.
Sep 23, 2026
Q: Multi-armed and contextual bandits, Bayesian optimisation, sequential Bayesian experimental design / deep adaptive design, dynamic treatment regimes with Q- and A-learning, switchback experiments and RLHF all choose actions from accumulating data. Laid out on one map, what is each optimizing, what is the state, and what feedback does it assume?
type/qa
topic/multi-armed-bandits
topic/bayesian-experimental-design
topic/causal-inference
topic/online-experimentation
topic/large-language-models
topic/machine-learning
Sep 23, 2026
Q: Does peeking (optional stopping) matter for a Bayesian? Reconcile the frequentist peeking problem, always-valid p-values / mSPRT and confidence sequences with the likelihood principle, Thompson sampling and sequential Bayesian experimental design.
type/qa
topic/bayesian-statistics
topic/online-experimentation
topic/research-methodology
topic/multi-armed-bandits
topic/calibration
Sep 23, 2026
Q: How does advertising carryover (adstock) violate the assumptions of switchback experiments, always-valid sequential tests and geo tests, and what design changes fix it?
type/qa
topic/online-experimentation
topic/market-response
topic/causal-inference
topic/time-series
topic/research-methodology
Sep 23, 2026
CUPED and Regression-Adjusted Variance Reduction
source/ingested
topic/research-methodology
topic/online-experimentation
topic/variance-reduction
type/method
doc/paper
Sep 23, 2026
Online Experimentation - Overview
source/ingested
topic/research-methodology
topic/online-experimentation
topic/causal-inference
type/overview
doc/paper
Sep 23, 2026
The Peeking Problem and Optional Stopping
source/ingested
topic/research-methodology
topic/online-experimentation
topic/sequential-analysis
type/concept
doc/paper
Sep 23, 2026
Online Experimentation - Index
type/index
source/ingested
topic/research-methodology
topic/online-experimentation
Sep 18, 2026
Q: CUPED, regression adjustment, double/debiased ML residualization and CausalImpact covariates all 'adjust for other variables'. When is adjustment for precision, and when is it for identification — and what goes wrong when the two are confused?
type/qa
topic/causal-inference
topic/econometrics
topic/online-experimentation
topic/treatment-effects
topic/market-response
Sep 18, 2026
Q: Exchangeability underlies permutation/randomization tests, conformal prediction and hierarchical priors. Which vault methods break when it fails (time series, covariate shift, interference, clustering), and what replaces it in each case?
type/qa
topic/causal-inference
topic/conformal-prediction
topic/econometrics
topic/bayesian-statistics
topic/online-experimentation
Sep 18, 2026
Always-Valid p-values and the mSPRT
source/ingested
topic/research-methodology
topic/online-experimentation
topic/sequential-analysis
topic/hypothesis-testing
type/method
doc/paper
Sep 18, 2026
Confidence Sequences
source/ingested
topic/research-methodology
topic/online-experimentation
topic/sequential-analysis
topic/concentration-inequalities
type/concept
doc/paper
Sep 18, 2026
Interference and Marketplace Experiments
source/ingested
topic/research-methodology
topic/online-experimentation
topic/interference
topic/causal-inference
topic/marketplaces
type/concept
doc/paper
Sep 18, 2026
Switchback Experiment Design and Analysis
source/ingested
topic/research-methodology
topic/online-experimentation
topic/interference
topic/experimental-design
topic/randomization-inference
type/method
doc/paper
Sep 18, 2026
Sample Ratio Mismatch and Trustworthiness Checks
source/ingested
topic/research-methodology
topic/online-experimentation
topic/data-quality
topic/selection-bias
type/application
doc/paper