Index: Bayesian Experimental Design

Bayesian Experimental Design

Routing Summary

Information-theoretic design of experiments: choose designs $\xi$ to maximize the expected information gain (EIG) about latents $\theta$. This topic ingests four papers tracing the field’s full arc — its foundation (Lindley 1956), fast EIG estimation (Foster 2019), unified gradient design optimization (Foster 2020), and a review through policy-based adaptive design (Rainforth 2023) — plus a fifth on the earn-while-learning sibling paradigm, multi-armed bandits (Russo et al. 2018). Contains 27 notes across 5 sub-topics.

• The big picture across all papers / where to start? → Bayesian Experimental Design - Overview
• Core concepts (EIG, nested estimation, adaptive design)? → Foundations
• Fast EIG estimators (posterior, marginal, VNMC, implicit)? → Variational EIG Estimators
• One-stage gradient design (ACE, PCE, high-D applications)? → Gradient-Based Unified BOED
• The state of the field (objectives, computation, policies, challenges)? → Modern BED Review
• Earning reward while learning online — Thompson sampling, UCB, regret bounds? → Multi-Armed Bandits and Thompson Sampling

Sub-topics

• Foundations — COVERS: Lindley’s (1956) founding average-information measure (Defs 1–2, Theorems 1–9, the design rule); the EIG objective & its four equivalent (mutual-information) forms; double intractability and the NMC estimator ($\mathcal{O}(C^{-1/3})$); sequential/adaptive design and the incremental/total EIG. (4 notes.)
• Variational EIG Estimators — COVERS (Foster 2019): four amortized variational EIG estimators — posterior/Barber–Agakov (lower), marginal (upper), VNMC (upper, consistent), implicit-likelihood — with $\mathcal{O}(T^{-1/2})$ convergence and selection rules. (6 notes.)
• Gradient-Based Unified BOED — COVERS (Foster 2020): single SGA loop jointly optimizing a variational lower bound and the design; the ACE & PCE contrastive bounds; likelihood-free ACE and gradient estimators; high-dimensional applications (400-D regression, 100-D docking). (5 notes.)
• Modern BED Review — COVERS (Rainforth 2023): EIG vs Fisher-information objectives; the computational revolution (MLMC debiasing, variational, implicit); stochastic-gradient design; deep adaptive design (policies); open challenges. (6 notes.)
• Multi-Armed Bandits and Thompson Sampling — COVERS (Russo et al. 2018): the Beta-Bernoulli/general TS algorithm; UCB/greedy alternatives and the Gittins index; Bayesian regret, the Lai–Robbins bound, eluder dimension, and the information-ratio analysis; linear/GLM/contextual bandit reward models; approximate posterior sampling (Laplace, Langevin, bootstrap, ensemble), nonstationarity, and PSRL/deep exploration in RL. (6 notes.)

Cross-Cutting Concepts

Concepts that span multiple sub-topics:

• Expected Information Gain — the single objective every method optimizes: Expected Information Gain (defined) → estimated by Foster 2019 estimators / MLMC & variational → optimized by Foster 2020 SGA / DAD policies.
• Mutual-information bounds repurposed for design — Barber–Agakov (Variational Posterior Estimator (Barber-Agakov)), InfoNCE (Prior Contrastive Estimation (PCE)), and the adaptive contrastive bound (Adaptive Contrastive Estimation (ACE)).
• Bound-trapping — pairing a lower bound (ACE/${\hat{\mu }}_{\text{post}}$) with an upper bound (VNMC/${\hat{\mu }}_{\text{marg}}$) to verify designs: Convergence Rates and Estimator Selection, High-Dimensional Design Applications.
• Sequential / adaptive design — from greedy BAD (Sequential and Adaptive BED) to amortized non-myopic policies (From Designs to Policies (Deep Adaptive Design)).
• Implicit-likelihood models — Implicit Likelihood Estimator (error-bounded) → Likelihood-Free ACE and Gradient Estimation (bound-preserving) → review §3.3.2.
• Learn vs. earn-while-learning — pure BED maximizes EIG with no reward at stake; bandit regret bounds use an analogous information ratio that divides squared regret by mutual information, making the EIG the limiting case of “cost per bit” as the reward term vanishes. See Q - BED vs Bayesian Optimization vs Bandits for Media Experimentation.

Concept Dependency Chain

Lindley’s Information Measure → Expected Information Gain → Nested Estimation and Nested Monte Carlo → Variational BOED - Overview → {Variational Posterior Estimator (Barber-Agakov), Variational Marginal Estimator, Variational NMC Estimator, Implicit Likelihood Estimator} → Convergence Rates and Estimator Selection → Unified SGD BOED - Overview → Adaptive Contrastive Estimation (ACE) → Prior Contrastive Estimation (PCE) / Likelihood-Free ACE and Gradient Estimation → High-Dimensional Design Applications; and (review thread) Information-Theoretic Design Objectives → The Computational Revolution in EIG Estimation → Optimization and Gradient Schemes for BED → From Designs to Policies (Deep Adaptive Design) → Open Challenges and Future Directions.

Sources

• Lindley 1956 - On a Measure of the Information Provided by an Experiment — Lindley, D.V., On a Measure of the Information Provided by an Experiment, Ann. Math. Stat. 27(4):986–1005, 1956. The founding paper.
• Foster et al 2019 - Variational Bayesian Optimal Experimental Design — Foster et al., Variational Bayesian Optimal Experimental Design, NeurIPS 2019. arXiv:1903.05480.
• Foster et al 2020 - Unified Stochastic Gradient BOED — Foster et al., A Unified Stochastic Gradient Approach to Designing Bayesian-Optimal Experiments, AISTATS 2020. arXiv:1911.00294.
• Rainforth et al 2023 - Modern Bayesian Experimental Design — Rainforth, Foster, Ivanova, Bickford Smith, Modern Bayesian Experimental Design, Statistical Science 2023. arXiv:2302.14545.

See Also

• Bayesian Statistics — inference, computation, and decision analysis that BED builds on
• Experimental Design (frequentist) — the classical power-analysis counterpart
• Decision Analysis — EIG as expected utility of an experiment
• Multi-Armed Bandits and Thompson Sampling — the earn-while-learning sibling paradigm (Thompson sampling, UCB, regret)
• Q - BED vs Bayesian Optimization vs Bandits for Media Experimentation — BED vs. Bayesian optimization vs. bandits, worked through a media-measurement lens