Experiment prioritisation (closes gap 7)
Tests are ranked by the single-test score below, but the portfolio of tests is chosen by a cost-aware greedy rule on EIG, and EVOI is computed for the chosen portfolio as a whole, because the near-optimality guarantee covers only EIG under a count limit.¶
Single-test score. Score(k) = λ · EIG + (1 − λ) · EVOI − γ · cost − ρ · risk, each term divided by its maximum over the candidate set, with the divisors recorded in the priority snapshot; EIG is measured on the decision-relevant parameters named in the snapshot. EVOI is computed by Monte Carlo over joint posterior draws against the explicit decision rule; the closed-form approximation is for intuition only. EVOI is computed from joint draws when the model declares joint_draws or joint_gaussian dependence. Otherwise it is computed under both couplings of G-DEP-01 (section 7) and reported as a range, labelled DEPENDENCE_ASSUMED with envelope_kind: range, never as a bound. Tier D uses library priors (section 8).¶
What the guarantee covers. Greedy selection has a constant-factor guarantee for a monotone submodular function under a cardinality constraint (Nemhauser, Wolsey and Fisher, 1978). EIG is submodular when test outcomes are conditionally independent given the parameters. EVOI is not submodular in general, because two tests can be worth more together than apart when only their combination flips a decision. And a budget limit is a knapsack constraint, where plain greedy loses the guarantee.¶
Portfolio rule (normative)¶
- A cluster is a set of channels whose absolute posterior correlation of response parameters exceeds a registered threshold (proposed 0.5). Channels in no cluster are singletons. Clusters define the correlation structure used to compute joint EIG. They are not separate budgets.¶
- Over all affordable candidates, under the one budget, build P1 by cost-scaled greedy selection on joint EIG. Build P2 as the single most informative affordable test.¶
- Keep whichever of P1 and P2 has the higher EIG. When test outcomes are conditionally independent given the parameters, and marginal EIG is monotone submodular and computed exactly, this construction has a constant-factor guarantee for EIG (Khuller, Moss and Naor, 1999; Leskovec et al., 2007). No bound is claimed for EVOI. Otherwise the snapshot records
NO_GUARANTEE. The EIG estimator and its Monte Carlo SE are recorded.¶ - Compute joint EVOI for the kept portfolio and for the top alternatives by Monte Carlo. If an alternative has higher joint EVOI, the measurement lead chooses and the choice is logged.¶
- Briefs MUST NOT claim the (1 − 1/e) bound for EVOI or for budget-constrained selection.¶
Inputs and outputs. A priority_snapshot artifact records the model version, joint draws used, scores, chosen portfolio and the reason for any override. It feeds the test board and the evidence.aging event loop (section 19).¶
Research items, non-blocking. Batch Bayesian design for many parallel tests, and amortised design once enough historical designs exist.¶