Guide · transportability
Why a model does not travel
Every check in the previous guides can pass, and the number can still be wrong where it lands. Identification asks whether an effect is recoverable from these data. Transportability asks the question nobody writes down: recoverable for whom, and does that match who you are about to decide for.
The situation
Nothing went wrong, and the answer is wrong
A household water-filter programme was evaluated in district A. Uptake was recorded per household, illness days were counted over twelve weeks, and identification was settled properly. The estimate is defensible: filters cut reported illness by -1.84 days per period, with a standard error of 0.41.
The decision is in district B. The ministry budgets per person, not per household, and plans over a year, not twelve weeks. District B is also older — a third of district A is under five, against a fifth of district B — and that matters, because filters help small children most.
Nobody is proposing anything dishonest. Somebody is about to put -1.84 into a slide about district B, and three separate things about that number are not true there.
This is the failure that survives every other check
A confounder shows up as a biased estimate you can hunt for. A bad prior shows up in a posterior predictive check. A transfer that was never written down shows up as nothing at all: the number is correct, the interval is honest, the code has no bug, and the answer is to a question nobody asked.
Step one
Name what differs, before arguing about it
A quantity in axiom is an Estimand,
and it is the same quantity as another only if all eight of its facets match.
Comparing the two here gives the argument its actual shape: three facets differ, and
they are not the same kind of problem as each other.
Written down, the disagreement stops being “is this number applicable” — a question nobody can win — and becomes three specific questions with three different owners. That is most of the value, and it happens before any mathematics.
Step two
A selection diagram: where two worlds differ
Populations always differ somehow. The question is never “are they different” — they are — but whether they differ in a way that changes this effect. That is a claim about mechanisms, so it is written on a graph.
The same causal structure, plus one node for the difference
Every variable whose mechanism differs between the two districts
gets an extra parent S[·], meaning “this one is not the same
here”. Here only the age composition differs.
axiom.identify.selection_diagram(graph) · graph hash 7d7407f973adf97c18706614ff786ef953d1f79f461aaf0dbcdfea29ab33f01d
Once the difference is a node, ordinary graph reasoning answers the
question. If the S-node can be separated from the outcome by conditioning on some
measured set Z, the source study's Z-specific effects
can be reweighted to the target's own distribution of Z. That set is
called S-admissible, and the reweighting is the
transport formula.
What axiom returns for this diagram
S-admissible set:
age — so district A's age-specific effects,
reweighted by district B's age mix. This is why the strata weights on
Population are not decoration.
The verdict is not the word “yes”
It is a formula, the set that licenses it, a
named assumption — s_admissibility — and the
thing that would refute it:
a mechanism difference between the populations the diagram omits. An answer you cannot
attack is not an answer you can trust.
Step three
Four things a graph can say
The same question, asked against four different beliefs about what differs between the districts. Only one is the easy case, and one of the others is better news than it looks.
s_admissible_adjustment.
age
trivial — and this is the
surprise. If the target population can identify the effect from its own data,
you do not need the source study at all. The right move is to stop arguing about
transfer and go and estimate it there.
age, sanitation
sanitation
sid_recursion
Why the last one does not say “no”
axiom implements three sufficient
conditions for transportability, not the complete decision procedure. When none of
them fires, a valid transfer may still exist that these three cannot see. Saying
“not transportable” would be a claim about the world; saying
unsupported with sid_recursion
attached is a claim about the evidence, which is the only one the library is
entitled to make.
Step four
What the assumptions cost
A graph settles one facet out of the three. Twelve weeks is not fifty-two weeks and a household is not a person, and no causal graph has an opinion about either. Each remaining facet is licensed by a named assumption, and each assumption comes with an operator that says what the correction actually is.
What the study measured
district A · per household · 12 weeks
What it becomes where the decision is
district B · per person · 52 weeks — the point estimate grew, and the standard error grew by a factor of 2.173
The widening is the honest part, and it is the part a spreadsheet never does. Reading a household-level measurement at the person level does not make the effect less certain by accident. It makes it less certain because five people in one house share a water source, and pretending they are five independent observations is exactly the mistake that produces intervals nobody should believe.
Step five
The thing you actually hand over
Not the number. The number, plus every boundary it crossed and what was assumed to get it across. That list is built by construction rather than by discipline, and it refuses to be incomplete.
rests on
s_admissibility — challenged by
overlap; moderator interaction in meta
satisfied
rests on
carryover_contained — challenged by
half-life against window length
unverified
rests on
linear_aggregation — challenged by
the Jensen gap under a nonlinear response
unverified
rests on
s_admissibility — challenged by
a mechanism difference between the populations the diagram omits
satisfied
Completeness
every differing facet accounted for exactly once (population, window, level) — a ledger that skipped a differing facet would fail this check rather than reaching a reader.
Overall status
Not identified, and correctly so:
2 of the 4 lines are
still unverified. Nothing here has checked the half-life against
the window or the Jensen gap against a nonlinear response. The status describes
the evidence; it is not a grade.
What to take away
Three habits, in order of how much they save you
Write down the question you need before you look at the one you have
Almost all of the value on this page came from the first step, which was free. Naming three differing facets turned an unwinnable argument about whether a number “applies” into three specific questions with three different owners.
Make the difference between populations a node, not an adjective
“Different population” is not a finding. S[age] is: it says
exactly which mechanism differs, it can be argued with, and once it is on the
graph the question of whether that difference matters has a mechanical
answer.
Let the correction widen the interval
Every honest transfer costs precision. If a number crosses three boundaries and comes out just as certain as it went in, the uncertainty did not vanish — it stopped being recorded. Here the standard error grew 2.173×, and that is the finding.
The Making a model travel tutorial runs this whole story as six executable steps, and the next guide is about the machinery that makes the ledger above something you can still trust six months later.