Q: When should continuous media learning use Bayesian experimental design vs Bayesian optimization vs a bandit?

When should continuous media learning use BED vs Bayesian optimization vs a bandit?

Summary

All three sit on the same Bayesian surrogate of the response surface and differ only in objective. Bayesian experimental design (BED) maximizes expected information gain about the parameters $\theta$ — use it when the deliverable is an accurate measurement (attribution, interaction effects, elasticities). Bayesian optimization (BO) maximizes an acquisition function to find the best budget allocation $x_{\text{*}}$ in as few expensive tests as possible — use it when the deliverable is a decision, not a report. A bandit maximizes cumulative reward / minimizes regret while learning online — use it for always-on tactical allocation (creative rotation, bidding) where every impression is both a test and a payout. Rule of thumb: learn → BED, optimize → BO, earn-while-learning → bandit. They compose: BED/BO to establish the response surface periodically, a bandit to exploit it continuously.

Answer

The one thing they share

Each method maintains a Bayesian posterior over the response surface (an MMM or a GP) and chooses the next action by optimizing an expected quantity under that posterior. What differs is what expectation they optimize:

Paradigm	Optimizes	Objective	Media deliverable
BED	expected information about $\theta$	$\mathrm{EIG}(\xi )={\mathrm{MI}}_{\xi }(\theta ;y)$ (Expected Information Gain)	accurate effects/interactions, attribution
BO	expected improvement in the objective	acquisition $\alpha (x\mid \mathcal{D})$ — EI/UCB/KG (Acquisition Functions)	the best media mix, found cheaply
Bandit	expected cumulative reward	minimize regret ${\sum }_t[f(x_{\text{*}})-f(x_t)]$	most revenue while allocating live

1. BED — when the goal is to learn

Use BED when the output is a measurement: you need trustworthy estimates of channel effects, saturation, and especially cross-channel interactions to brief planning. BED explicitly targets the reduction of parameter uncertainty (Bayesian Experimental Design - Overview), and the EIG is “the most common and best-performing” design objective (Information-Theoretic Design Objectives). Concretely, this is the geo-holdout-as-design workflow of Q - Encoding a Geo-Holdout as a Bayesian Experimental Design and Computing Its EIG. For continuous programs, adaptive BED and amortized DAD policies pick each successive test.

2. BO — when the goal is to optimize

Use BO when you don’t need the whole response surface, just its argmax: the budget split that maximizes revenue/ROAS with the fewest expensive experiments. The Global Optimisation Problem is exactly “find the global minimiser of an expensive, noisy black-box under an exploration–exploitation trade-off.” BO’s acquisition functions — PI, EI, GP-UCB, Knowledge Gradient — encode that trade-off in closed form; Value Loss and Entropy Search adds information-theoretic acquisitions that value learning the location of the optimum specifically. Media-relevant machinery lives in Further Topics in Global Optimisation: batch/parallel BO (launch several geo-tests at once) and multi-fidelity (blend cheap correlational reads with expensive clean experiments).

BED and BO are two settings of one dial

BED maximizes information about all of $\theta$; entropy-search BO maximizes information about the optimum $x_{\text{*}}$ only. Choose BED to characterize the surface, entropy-search BO to shortcut straight to the best allocation. This is the unifying value-of-information insight from Q - Continuous Learning in Media Measurement with Interaction Effects.

3. Bandits — when the goal is to earn while learning

Use a bandit for always-on tactical decisions — creative/message rotation, real-time bidding, on-site placement — where each action simultaneously is the experiment and earns (or costs) reward, so you pay for every unit of exploration as regret. Bandits (ε-greedy, UCB, Thompson sampling) balance exploration and exploitation to minimize cumulative regret rather than to end with a precise parameter estimate. This paradigm now has a dedicated home — Multi-Armed Bandits and Thompson Sampling - Overview (Russo et al. 2018) — connecting to existing vault content via three anchors:

• Thompson sampling is the practical default: draw one posterior sample per period and act greedily on it. Bernoulli Bandit and Thompson Sampling Algorithm gives the exact Beta-Bernoulli conjugate-update algorithm (and its general non-conjugate form); Contextual and Linear Bandits extends it to the linear/GLM/contextual reward models an MMM-driven allocator would actually use; Approximate Thompson Sampling and Practical Extensions covers nonstationary drift and Laplace/Langevin/bootstrap approximations when conjugacy fails.
• Shared vocabulary with BO: the bandit UCB rule (UCB and Greedy Algorithms for Bandits) is the same idea as GP-UCB in Acquisition Functions — an upper-confidence-bound acquisition. BO is essentially a bandit over a continuous, GP-modeled arm space; Further Topics in Global Optimisation notes BO “relates to but differs from reinforcement learning.”
• Sequential decision theory: choosing actions over time from accumulating history to maximize a long-run objective is the dynamic treatment regime problem — Q- and A-learning - Overview, Optimal Regime via Dynamic Programming (backward induction, Q/value functions). A non-myopic media-allocation policy is a DTR/RL problem; DAD is its BED-flavored, information-seeking cousin, and posterior-sampling RL (Approximate Thompson Sampling and Practical Extensions) is its bandit-flavored one.

4. How to choose — and compose

• Deliverable is a report/attribution (what does each channel and interaction do?) → BED.
• Deliverable is a one-shot or periodic allocation with few, expensive tests → BO.
• Deliverable is continuous live allocation where exploration has direct cost → bandit.
• Myopic vs long-horizon: greedy EIG/EI are one-step; use DAD policies (BED) or DTR/RL (bandit side) when tests are frequent and you must be non-myopic.

They are layers, not rivals: run BED/BO periodically to (re)learn the response surface and its interactions, then let a bandit exploit that surface continuously between refreshes.

Practical Implications

• Match the tool to the deliverable, not the algorithm’s popularity: measurement→BED, decision→BO, live money→bandit.
• Reuse one surrogate (MMM/GP) across all three; only swap the objective.
• Watch the cost of exploration: BED/BO treat experiments as an investment in information; bandits charge exploration to the P&L as regret in real time.

Source Notes

Note	Relevance
Bayesian Experimental Design - Overview · Expected Information Gain	The “learn” objective (EIG / mutual information)
Information-Theoretic Design Objectives	EIG as the best design objective; vs Fisher information
Bayesian Optimisation · Acquisition Functions	The “optimize” objective; PI/EI/UCB/KG closed forms
Value Loss and Entropy Search	Information-theoretic acquisitions (ES/PES/MES)
The Global Optimisation Problem	Exploration–exploitation, regret framing
Further Topics in Global Optimisation	Batch/multi-fidelity BO; BO vs RL
From Designs to Policies (Deep Adaptive Design)	Non-myopic amortized policies (BED side)
Q- and A-learning - Overview · Optimal Regime via Dynamic Programming	Sequential-decision / DTR analog of bandits/RL
Multi-Armed Bandits and Thompson Sampling - Overview · Bernoulli Bandit and Thompson Sampling Algorithm	The “earn while learning” objective; posterior-sampling algorithm
UCB and Greedy Algorithms for Bandits	Optimistic (UCB) alternative to sampling; Gittins index
Regret Bounds for Thompson Sampling	Formalizes regret; Lai-Robbins, eluder-dimension, and information-ratio bounds
Contextual and Linear Bandits · Approximate Thompson Sampling and Practical Extensions	Linear/GLM/contextual reward models; nonstationarity; PSRL

Related Concepts

• Decision Analysis — expected-loss framing shared by all three
• High-Dimensional Design Applications — scaling the “learn” objective to many dimensions
• Q - Continuous Learning in Media Measurement with Interaction Effects — the loop these three objectives can each drive
• Q - Uncovering Causal Estimates from Non-Experimental Data — when experiments/bandits aren’t available

Gaps

• No dedicated note on multi-armed bandits, Thompson sampling, UCB1, or contextual bandits. Resolved (2026-07-03): Russo et al. (2018), A Tutorial on Thompson Sampling, ingested as Multi-Armed Bandits and Thompson Sampling (6 notes: overview, algorithm, UCB/greedy alternatives, regret bounds, contextual/linear bandits, approximate sampling & practical extensions).
• No note on regret bounds (cumulative vs. Bayesian regret). Resolved: see Regret Bounds for Thompson Sampling (Lai-Robbins asymptotic bound, instance-independent and eluder-dimension bounds, information-ratio analysis).
• BO↔RL relationship is only mentioned in passing in Further Topics in Global Optimisation; the bandit-side analog, PSRL/deep exploration, is now covered in Approximate Thompson Sampling and Practical Extensions, but a note bridging BO and RL directly is still missing.
• Still open: no note formalizes information-directed sampling (Russo & Van Roy 2014a/2018a) as a standalone method beyond its mention in Regret Bounds for Thompson Sampling — worth a dedicated note if the vault ingests that line of work further.

Follow-Up Questions

• What would a contextual bandit for creative rotation look like with an MMM/GP as the reward model?
• How do you hand off from a BED/BO “learning” phase to a bandit “earning” phase without discarding the posterior?
• Is Thompson sampling on a GP surrogate equivalent to a randomized entropy-search acquisition?