Pillar 05 · axiom.meta
What does all the evidence say?
One study is an anecdote with a standard error. A programme accumulates dozens, from different teams, on different scales, some of them randomized and some of them model-based reads that everyone quietly trusts less. Pooling that is not averaging. It is deciding what is comparable, how much the studies really disagree, and whether one kind of evidence runs systematically high.
Before any arithmetic
Refusing to pool what is not comparable
Eleven studies report a dimensionless standardized contrast. A twelfth reports the same quantity per acre. Averaging the twelve is a category error, and axiom will not do it silently.
Unsupported — the corpus is refused, with the reason
“study 's12' reports 'standardized_contrast' on a scale (dimension 1, unit_scale 'per_acre'); pooling is over dimensionless quantities and no TransferPlan supplied”
The way through is not a flag that turns the check off.
It is to supply a TransferPlan — the same object the
calibration layer uses — which states how the per-acre
reading maps onto the dimensionless one and carries that claim as a hashed record.
Once the plan is attached, the study is admitted with its provenance intact, and
the pool runs over 12 studies.
The corpus
Twelve studies, two kinds of evidence
Of the twelve, 8 are experiments and 4 are model-based reads. The forest plot below is coloured by which — because that distinction turns out to be the whole story.
Every study, with the pooled estimate
Marker size is the study’s weight in the pool.
axiom.meta.forest_data(corpus, None, pooled, mass=0.95) · pooled by random_effects(y, se, tau_method="reml", knapp_hartung=True)
Pooled, treating all twelve alike
95% · REML with the Knapp–Hartung adjustment · se 0.0286
| method | pooled mean | se | τ² | 95% interval |
|---|---|---|---|---|
| DerSimonian–Laird | 0.7025 | 0.0285 | 0.00282 | [0.647, 0.758] |
| Paule–Mandel | 0.7023 | 0.0286 | 0.00287 | [0.646, 0.758] |
| REML | 0.7024 | 0.0285 | 0.00284 | [0.647, 0.758] |
The finding
Model reads run high, and pooling naively buries it
The classical pool has no way to express “these four studies are a different kind of evidence”. axiom’s Bayesian pool carries an explicit bias term for the model-based reads, and first checks whether that term is identified at all — it is only identified because two contributors reported both kinds, which is what pins the two scales together.
Funnel plot, coloured by kind of read
Precise studies sit at the top. The separation by colour, not the asymmetry, is what matters here.
axiom.meta.funnel_data(y, se, pooled) with 90/95/99% contours · Egger intercept -2.904, p = 0.008
Experiment-scale mean, with the bias term carried
90% equal-tailed · versus 0.7024 from the classical pool that treats every read alike
How far model reads sit above experiments
± 0.0473 — identified: True, via lab_a, lab_b, the contributors who reported both kinds
The difference is not cosmetic
Pooling everything together gives 0.702. Separating the two kinds of read gives an experiment-scale mean of 0.627. If the number is going to be used to set a prior for the next study, the second is the one you want — and the gap between them is roughly the size of the effect people would be arguing about.
Diagnostics
Which studies are driving the answer
Two views of influence, because “drop it and see” and “how much heterogeneity does it contribute” are different questions and can disagree.
Baujat: heterogeneity contributed against influence on the pool
Top-right is a study that both disagrees with the others and moves the answer. Labels are shown for the studies that matter.
axiom.meta.baujat(y, se) under fixed-effect pooling
Leave-one-out influence
How far the pooled estimate shifts, in pooled-standard-error units, when each study is removed. Every model read pulls the pool up; every experiment pulls it down.
axiom.meta.leave_one_out(y, se, method="reml") · orange bars are model reads, blue are experiments
Two ways out
Hand it forward, or publish it safely
As a prior for the next study
The pooled result is not a full stop. It converts into a prior on the amplitude of the next dose–response fit, carrying a handoff ledger line so the eventual model records where its prior came from.
That is the loop closing: the surface that produced each study’s estimate is the same object that now receives the pooled prior.
As a published figure, under a privacy budget
When the corpus spans contributors who will not share raw results, axiom checks a release policy — minimum contributor count, no single contributor dominating — then adds calibrated noise and charges the privacy budget.
What is in axiom.meta
70 public symbols. Corpus construction and normalization, three τ estimators, Knapp–Hartung and prediction intervals, Egger and Baujat and leave-one-out, the Bayesian pool with its bias term, moderator designs, prior handoff, and the privacy release path with its epsilon ledger.
See all five together
The case study runs one trial from the causal graph through sizing, the dose–response, the sequential design, and the review that stops an arm — using every pillar on this tour.