Heath 2015 - Net Benefit, EVPI and EVPPI Definitions

Summary

Defines the monetary net benefit of each decision option, the “known-distribution” net benefit $N{B}_t(\theta )$ that varies with the uncertain parameters, and the EVPPI as the expected gain from learning a parameter subset $\varphi$ exactly (Eq. 1). EVPI is the special case $\varphi =\theta$. Both are non-negative, and EVPI bounds every EVPPI.

Overview

Health-economic models choose among $T+1$ options, $t=0$ (current standard) and $t=1,\dots ,T$ (new options; in practice rarely $T>5$). Each option has an effect $e_t$ and cost $c_t$ drawn from $p(e,c\mid \theta )$, where $\theta$ indexes uncertain model parameters. The paper (Sec. 2) builds VoI from this setup. The same machinery applies to any decision where benefit and cost are commensurable, for example incremental profit from a budget move.

Main Content

Monetary net benefit

For willingness to pay $k$ per unit of effect, option $t$ has net benefit $n{b}_t=k{e}_t-c_t$. Under full Bayesian uncertainty the expected net benefit is $N{B}_t=k\mathbb{E}[e_t]-\mathbb{E}[c_t]$, where the expectation is over the joint $p(e,c,\theta )=p(e,c\mid \theta )p(\theta )$. The currently optimal option attains $N{B}^{\ast }={\max }_{t}N{B}_t$.

Known-distribution net benefit

$N{B}_t(\theta )=k\mathbb{E}[e_t\mid \theta ]-\mathbb{E}[c_t\mid \theta ]$, the expectation taken over individual variability only (the term follows Baio, 2012). $N{B}_t$ is a number, but $N{B}_t(\theta )$ is a random variable because $\theta$ is uncertain, and ${\mathbb{E}}_{\theta }[N{B}_t(\theta )]=N{B}_t$.

Parameter partition

Write $\theta =(\varphi ,\psi )$: $\varphi$ are the parameters of interest (those further research could investigate) and $\psi$ the remaining nuisance parameters.

Expected Value of Partial Perfect Information (Eq. 1)

$$\text{EVPPI}={\mathbb{E}}_{\varphi }\left[\underset{t}{\max }{\mathbb{E}}_{\psi \mid \varphi }\right[N{B}_t(\theta )\left]\right]-\underset{t}{\max }{\mathbb{E}}_{\theta }[N{B}_t(\theta )].$$

The first term is the value of deciding with $\varphi$ known exactly (marginalising $\psi$ given $\varphi$) and then averaging over what $\varphi$ might turn out to be; the second is the value of deciding now.

Expected Value of Perfect Information (EVPI)

The case $\varphi =\theta$ (no nuisance parameters): $\text{EVPI}={\mathbb{E}}_{\theta }[{\max }_{t}N{B}_t(\theta )]-{\max }_t{\mathbb{E}}_{\theta }[N{B}_t(\theta )]$. The paper calls EVPI the “(overall)” quantity that is “easy to calculate numerically (but not analytically)” from PSA draws; it places an upper bound on the cost of any research that reduces uncertainty.

Ordering (standard result, proof sketch)

For every partition, $0\le \text{EVPPI}(\varphi )\le \text{EVPI}$. Sketch. Lower bound: ${\mathbb{E}}_{\varphi }[{\max }_t{g}_t(\varphi )]\ge {\max }_t{\mathbb{E}}_{\varphi }[g_t(\varphi )]={\max }_{t}N{B}_t$ with $g_t(\varphi )={\mathbb{E}}_{\psi \mid \varphi }[N{B}_t]$ (the max of expectations never exceeds the expectation of the max). Upper bound: $g_t(\varphi )\le {\mathbb{E}}_{\psi \mid \varphi }[{\max }_{s}N{B}_s(\theta )]$ for every $t$, hence ${\mathbb{E}}_{\varphi }[{\max }_t{g}_t]\le {\mathbb{E}}_{\theta }[{\max }_{s}N{B}_s(\theta )]$. The paper states Eq. (1) but not this ordering explicitly.

Why EVPPI, not the probability of cost-effectiveness

The introduction argues that the probability that an option is cost-effective “can dramatically overstate decision sensitivity”: if the cost of a wrong decision is small, there is “little value in agonising about the decision, even for a low probability of cost-effectiveness”. VoI weights the chance of being wrong by the size of the loss. Two limitations the paper names: VoI is theoretically unbounded and hard to interpret, and it can be computationally costly unless restrictive assumptions are made.

Examples

Two options, one uncertain gain. Let option 0 have $N{B}_0=0$ and option 1 have $N{B}_1(\theta )\sim N(m,s^2)$ with $m=0.3$, $s^2=0.5$. Deciding now picks option 1 (value $0.3$). EVPI $=\mathbb{E}[\max (N{B}_1,0)]-0.3=s\phi (m/s)+m\Phi (m/s)-m\approx 0.157$. If $N{B}_1=0.3+0.5\varphi +0.5\psi$ with $\varphi ,\psi$ independent standard normal, then EVPPI for $\varphi$ equals the same formula with sd $0.5$: $\approx 0.084$ (see the verified code in Heath 2015 - Regression-Based EVPPI (GAM and Gaussian Process)). So learning $\varphi$ alone captures about 54 percent of the total decision value.

Connections

• Relevance to marketing measurement: replace $k{e}_t-c_t$ with incremental profit of a reallocation; $\varphi$ is, for example, a channel’s saturation parameter and $\psi$ the rest of the MMM. EVPPI says whether an experiment on $\varphi$ could change the allocation at all.
• Related: Decision Analysis, From Inference to Decision, Expected Information Gain.

See Also

• Value of Information Concept Note - EVPI, EVPPI and EVSI for EVSI and EVPI as a price ceiling.
• Heath 2015 - Review of Expected Value of Information Methods - Overview