One question, carried all the way · step 5 of 8
The experiment lands
What did it measure, and does the model we already had agree?
The trial runs. Its result arrives as one typed Measurement — the estimand it measured, the number, the interval, and the conditions it was measured under — rather than as a slide with a percentage on it.
Before folding it in, ask whether the old model predicted it. agreement compares what the observational fit expected against what the experiment saw, and this is where the tutorial turns: they disagree.
step 5
from axiom.calibrate import Measurement, agreement
# What the world would really have shown. The analyst does not get to
# see this; the experiment reads it through noise.
diff = world.forward({"a": DOSE}) - world.forward({"a": NO_DOSE})
truth_experiment = float(np.mean(diff[:, 0]))
se_experiment = 0.35
measurement = Measurement(
estimand=experiment_estimand,
estimate=truth_experiment,
se=se_experiment,
mass=0.9,
source="depot-maintenance-rct-2026Q2",
)
print(f"the trial read {measurement.estimate:+.3f} "
f"{measurement.interval}")
says = agreement(biased, measurement, seed=SEED)
print()
print(f"the model we already had expected {says.posterior_mean:+.3f}")
print(f"the trial says {says.estimate:+.3f}")
print(f"agreement : {says.verdict} (z = {says.z:.1f})")
what it printed
the trial read +2.906 [2.33, 3.48] (90% WALD) the model we already had expected +4.554 the trial says +2.906 agreement : disagrees (z = -2.2)
Every line above was captured by running this code at build time. The steps share one namespace, so the code on this page is the code you would type after the steps before it.