Guide · writing the question down

The estimand, in eight facets

“The effect of filters on illness” is not a quantity. It is a topic. Somewhere between that phrase and a number, eight separate decisions get made — and if they are made silently, they get made differently by each person in the room.

Why bother

The argument nobody can win

Two analysts study the same programme and report different effects. The meeting that follows is about methods: whose model, whose covariates, whose software. It goes badly, because that is not what differs.

One computed the effect of going from no filter to one filter, per household, over the twelve weeks observed, in the district studied. The other computed the effect per person, at steady state, in the district being decided for. Both are correct. They are answers to different questions, and no comparison of methods will ever reveal that.

An Estimand is that question written down completely enough that the two are visibly different objects — with different content hashes — before anybody argues about the number.

The rule

Two quantities are the same quantity if and only if all eight facets match. Not “are roughly comparable”. Match. Anything else is a transfer, and a transfer needs a licence.

It is a key, not paperwork

The eight facets exist because they are exactly the dimensions along which an answer can fail to carry to a new setting. That is why Estimand is the object that transportability is defined on, and why the list has eight entries rather than however many felt tidy.

The eight

What each one fixes, and what it takes to change it

The table below is not transcribed from documentation. Each row was produced by taking one estimand, changing only that facet, and asking axiom what happens — so it cannot describe a rule the library no longer has.

quantity which functional: a contrast, a marginal, a ratio, an elasticity, an area
a different functional is a different quantity; re-express first blocked

intervention what is set, to what value, and how
licensed by surface_correct_between_doses — challenged by curvature; the chord-versus-marginal correction downgraded

outcome which outcome, at what aggregation, in what dimension
licensed by outcome_commensurability — challenged by usually not falsifiable; must be asserted downgraded

population the target population and the strata that define it
licensed by s_admissibility — challenged by overlap; moderator interaction in meta downgraded

window the time window and whether it is per-period or cumulative
licensed by stationary_dynamics and carryover_contained — challenged by half-life against window length downgraded

level the unit of analysis and the interference model
licensed by linear_aggregation — challenged by the Jensen gap under a nonlinear response downgraded

conditioning the strata the answer is conditional on; empty for a marginal one
licensed by target_strata_weights_known — challenged by the provenance of the strata distribution downgraded

dimension derived from the others and asserted against the declaration
quantities of different dimension are different quantities blocked

6 of the 8 can be bridged by naming an assumption. 2 cannot be bridged at all, and that is the more interesting number.

The two that cannot be bridged

Refusal is a feature, and it is rare enough to matter

A different functional is a different question

A contrast — how much does illness change between no filter and one filter — and an elasticity — what percentage change follows a one percent change in coverage — are not two versions of one quantity. There is no assumption that turns one into the other, so axiom does not offer one: a different functional is a different quantity; re-express first

The fix is not an argument. It is to compute the elasticity from the fitted surface, which is a thing you can do — but by re-deriving it, not by relabelling the contrast you already have.

A different dimension is a different thing

Illness days and treatment cost are not commensurable at any exchange rate the library is entitled to invent: quantities of different dimension are different quantities

Converting between them is a decision about value, and it belongs to whoever owns the decision — recorded explicitly, not smuggled in by a model that quietly added a currency to an outcome. See dimensions, units and scope.

The habit

Write the target estimand before you look at any data

It stops the wrong argument

When two answers disagree, the first question becomes “are these the same estimand?” rather than “whose model is better”. Often they are not, and the disagreement dissolves in a minute instead of a fortnight.

It is a commitment device

Written first, hashed, and stored, the estimand is evidence that the question was not adjusted after the answer arrived. That is most of what pre-registration buys, at almost no cost. See provenance.

It makes reuse safe instead of tempting

Six months later, somebody will find your number and use it. With an estimand attached, transfer_to tells them in one call what would have to be true — and refuses when nothing would be. Without one, they will just use it.

The Making a model travel tutorial builds exactly this comparison in its first step, and the transportability guide carries it all the way to a corrected number and its ledger.

Guide 5 of 12

← Identification comes before estimation All the guides Dimensions, units and scope →