One question, carried all the way · step 6 of 8
Folding the experiment into the model
What does the model say once it has to honour the measurement?
Refit the surface under the constraint that it reproduce what the trial saw. The result is a model that agrees with the experiment on the experiment's own terms — and can then be asked the decision's question, which the experiment never measured directly.
Every step of that transfer is written to a ledger: the window it crossed, the level it aggregated to, the carryover it assumed. The decision's number is reported with the ledger attached or it is not reported.
from axiom.calibrate import fit_calibrated
calibrated = fit_calibrated(
spec, observed_panel, [measurement], backend="laplace", draws=1000, seed=1
)
after = realize(
decision_estimand, calibrated, assume_identified=True, mass=0.9, seed=SEED
)
after_per_depot = after.summary.mean / REGION_DEPOTS
print("the decision's estimand, per depot-week")
print(f" observational : {before_per_depot:+.2f}")
print(f" calibrated : {after_per_depot:+.2f}")
print(f" break-even : {BREAK_EVEN:+.2f}")
print()
said = "fund it" if before_per_depot > BREAK_EVEN else "do not fund it"
says_now = "fund it" if after_per_depot > BREAK_EVEN else "do not fund it"
print(f"the observational model said : {said}")
print(f"the experiment says : {says_now}")
the decision's estimand, per depot-week observational : +7.16 calibrated : +6.12 break-even : +6.75 the observational model said : fund it the experiment says : do not fund it
Every line above was captured by running this code at build time. The steps share one namespace, so the code on this page is the code you would type after the steps before it.