Challenges · C14 · ★★★★☆

Small for gestational age: whose chart flags whom?

Difficulty
★★★★☆
Time
4-5 hours
Data
A random sample of 20,000 of the 3.49 million singleton births in the United States in 2023 (NCHS natality public-use file: birth weight, two gestational-age estimates, sex, smoking, gestational hypertension)
Skills
Unknown codes and competing measurements (two gestational-age estimates) · cleaning rules that must not touch the tail you care about · smooth quantile curves with a random-walk (discrete P-spline) prior · shrinking a sex difference where data are thin · the asymmetric Laplace as a working likelihood, and checking its bands against the bootstrap and on held-out births · a bootstrap-calibrated generalised (Gibbs) posterior · conditional vs population centile charts · turning two posteriors into "babies per 1000 flagged" with uncertainty, and a recommendation

The brief

A hospital network is replacing its newborn-assessment software. Every baby is weighed at birth, and a baby lighter than the 10th centile of birth weight for its gestational age and sex is flagged small for gestational age (SGA): it gets extra glucose checks, a paediatric review and closer follow-up, because SGA babies carry a higher risk of stillbirth, hypoglycaemia and later problems. The network's obstetric lead asks you:

"Build us the 10th-centile chart, by sex, from recent US births - with an honest statement of how sure you are of it, especially for the preterm weeks. And a question our midwives keep asking: should babies of mothers who smoked, or who had gestational hypertension, be judged against their own charts? Those babies are smaller anyway. How many babies per 1000 would that stop - or start - flagging, and would you recommend it?"

This is the sequel to example E83 (Bayesian quantile regression). Do E83 first: this challenge assumes you know the check loss, why the asymmetric Laplace is only a working likelihood, and the sandwich / learning-rate fixes. E83 asked what quantile regression can tell you; here a chart has to be used, by sex, at every week from 24 to 42, and a policy question rides on it.

How this notebook works

  • Each task states what to deliver, not how. Write your code in the YOUR CODE HERE cells.
  • Stuck? h.hint("task2") reveals hints one level at a time: nudge → approach → code skeleton. Try to get by on nudges.
  • h.check("task2", q10_40_girls=...) compares your numbers with the reference solution.
  • A full worked solution lives in notebooks/solutions/. Open it only when you are done (or truly stuck).
import logging
import time
import warnings

import arviz as az
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import pymc as pm
import pytensor.tensor as pt

from pymc_challenges import Hints, data

RANDOM_SEED = 2023
rng = np.random.default_rng(RANDOM_SEED)
az.style.use("arviz-variat")
logging.getLogger("pymc").setLevel(logging.WARNING)  # several fits: no sampler banner per fit
warnings.filterwarnings("ignore", category=RuntimeWarning, module="arviz")
pd.set_option("display.width", 160)
T_START = time.time()

h = Hints("C14")
h.tasks()

C14 - Small for gestational age: whose chart flags whom?

  • task0 - Which gestational age? Which records? (3 hints, 1 check(s))
  • task1 - The quick chart (3 hints, 1 check(s))
  • task2 - A smooth chart that borrows strength (3 hints, 2 check(s))
  • task3 - Can you trust the bands? (3 hints, 2 check(s))
  • task4 - Charts for smokers and for gestational hypertension (3 hints, 2 check(s))
  • task5 - The decision (3 hints, 2 check(s))

The data

One row per singleton birth. load only parses the raw NCHS codes; it does not clean them.

column meaning
dbwt birth weight in grams (9999 = unknown)
combgest gestation in completed weeks, combined estimate: from the date of the last menstrual period (LMP), with the obstetric estimate filled in when the LMP date is missing or implausible (99 = unknown)
oegest_comb gestation in completed weeks, obstetric estimate recorded by the clinician (usually from an early ultrasound) (99 = unknown)
sex M / F
cig_0 ... cig_3 cigarettes per day before pregnancy and in trimesters 1, 2, 3 (99 = unknown)
rf_ghype gestational hypertension: Y / N / U (unknown)
others mother's age, race, education, BMI, weight gain, diabetes ... (see data.describe)
data.describe("natality_births")
raw = data.load("natality_births")
raw[["dbwt", "combgest", "oegest_comb", "sex", "cig_1", "cig_2", "cig_3", "rf_ghype"]].describe(include="all").T
natality_births: A simple random sample of 20,000 of the 3,491,735 SINGLETON births of 2023 (raw codes; 9 / 99 / 99.9 / 999 / 9999 = unknown, empty = not reported): dob_mm, mager (mother's age), mrace6 (1 White, 2 Black, 3 AIAN, 4 Asian, 5 NHOPI, 6 more than one race), dmar (1 married, 2 unmarried; not reported for California), meduc (1-8 education, 9 unknown), tbo_rec (total birth order), previs (prenatal visits), cig_0..cig_3 (cigarettes per day before pregnancy and in trimesters 1-3), m_ht_in (inches), bmi (pre-pregnancy), pwgt_r (pre-pregnancy weight, lb), wtgain (lb), rf_pdiab / rf_gdiab / rf_ghype (pre-pregnancy diabetes, gestational diabetes, gestational hypertension: Y/N/U), sex (M/F), combgest and oegest_comb (gestation in weeks, combined and obstetric estimate), dbwt (birth weight, grams).
  source: National Center for Health Statistics (NCHS), National Vital Statistics System, Natality public use file 2023 (all 3,605,081 US birth certificates of 2023; codes as in the NCHS 'User Guide to the 2023 Natality Public Use File'), read from NBER's CSV conversion (the URL; 188 MB). US government work, public domain; NCHS asks users to cite the source and not to attempt to identify anyone. BUILT by tools/build_e83_natality.py (a seeded random subsample) - keep the cached file.
  url:    https://data.nber.org/nvss/natality/csv/2023/natality2023us.zip
count unique top freq mean std min 25% 50% 75% max
dbwt 20000.0 NaN NaN NaN 3277.88435 593.076428 229.0 2976.0 3300.0 3629.0 9999.0
combgest 20000.0 NaN NaN NaN 38.66105 3.007759 17.0 38.0 39.0 40.0 99.0
oegest_comb 20000.0 NaN NaN NaN 38.4858 2.685286 17.0 38.0 39.0 39.0 99.0
sex 20000 2 M 10214 NaN NaN NaN NaN NaN NaN NaN
cig_1 20000.0 NaN NaN NaN 0.64555 5.98433 0.0 0.0 0.0 0.0 99.0
cig_2 20000.0 NaN NaN NaN 0.564 5.834981 0.0 0.0 0.0 0.0 99.0
cig_3 20000.0 NaN NaN NaN 0.92785 8.478664 0.0 0.0 0.0 0.0 99.0
rf_ghype 20000 3 N 17989 NaN NaN NaN NaN NaN NaN NaN

Task 0 · Which gestational age? Which records?

A chart is only as good as the gestational ages it is built on, and the file gives you two.

Deliver

  1. The unknown codes that matter for a weight-for-age chart, and how many records they remove.
  2. Evidence for choosing between combgest and oegest_comb: how often do they disagree, by how much, and what would each do to a weekly 10th centile - at the preterm weeks and after 41 weeks? Report the share of births with a known weight that combgest places at 43 weeks or more.
  3. A rule for records whose weight is implausible for their gestational age, and an argument for which side(s) of the distribution that rule may touch, given what the chart is for.
  4. The range of weeks the chart can cover.
Your turn Work this task in the notebook.

The analysis data (everyone uses this from here on)

So that numbers can be compared, the rest of the notebook uses the rules above: obstetric gestational age, 24-42 weeks, unknown weight / gestation / smoking (first or second trimester) / hypertension dropped, and the upper-side-only, preterm-only trim. A smoker smoked in any trimester (a third-trimester "unknown" counts as no information, not as a reason to drop: there was no third trimester for many preterm babies, see E83). We model log birth weight: quantiles are equivariant under monotone transforms, so the 10th centile of log weight is the log of the 10th centile, and effects become percentages.

The births are split at random into a training set of 10,000 for fitting the chart and a held-out set for checking it and for the decision.

ok = ((raw.dbwt < 9999) & (raw.oegest_comb < 99) & (raw.cig_1 < 99) & (raw.cig_2 < 99)
      & (raw.rf_ghype != "U") & raw.oegest_comb.between(24, 42))
births = raw[ok].copy()
births["lbw"] = np.log(births.dbwt / 1000)  # log kg
grp = births.groupby(["oegest_comb", "sex"]).lbw
zrob = (births.lbw - grp.transform("median")) / grp.transform(lambda v: 1.4826 * np.median(np.abs(v - np.median(v))))
births = births[~((zrob > 4) & (births.oegest_comb < 37))].reset_index(drop=True)
births["boy"] = (births.sex == "M").astype(int)
births["week"] = births.oegest_comb.astype(int)
births["smoker"] = ((births.cig_1 > 0) | (births.cig_2 > 0) | ((births.cig_3 > 0) & (births.cig_3 < 99))).astype(int)
births["hyp"] = (births.rf_ghype == "Y").astype(int)

WEEKS = np.arange(24, 43)
N_WEEKS = len(WEEKS)
SEXES = ["girl", "boy"]
N_TRAIN = 10_000
perm = rng.permutation(len(births))
train = births.iloc[perm[:N_TRAIN]].reset_index(drop=True)
test = births.iloc[perm[N_TRAIN:]].reset_index(drop=True)
y_tr, wk_tr, sx_tr = train.lbw.to_numpy(), train.week.to_numpy() - 24, train.boy.to_numpy()
y_te, wk_te, sx_te = test.lbw.to_numpy(), test.week.to_numpy() - 24, test.boy.to_numpy()
print(f"{len(births):,} births: training {len(train):,}, held out {len(test):,}; "
      f"smokers {births.smoker.mean():.1%}, gestational hypertension {births.hyp.mean():.1%}")
counts = pd.crosstab(train.sex, train.week)
counts
19,852 births: training 10,000, held out 9,852; smokers 3.2%, gestational hypertension 9.9%
week 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
sex
F 3 2 5 8 7 5 7 8 19 24 60 61 168 595 819 1918 906 256 11
M 4 6 6 4 3 10 11 8 21 26 61 91 190 619 950 1998 867 230 13

Task 1 · The quick chart

The obvious chart: for each week and sex, the empirical 10th centile of the training births.

Deliver

  1. The chart (both sexes, all weeks 24-42) on a scale where you can see the preterm weeks.
  2. What is wrong with it, in words a neonatologist would accept: look at neighbouring weeks and at the difference between boys and girls where there are few births.
  3. The share of held-out babies it flags, by gestational-age band and sex (24-31, 32-34, 35-36, 37, 38, 39, 40, 41-42 weeks). Is 10% what you get?
Your turn Work this task in the notebook.

Task 2 · A smooth chart that borrows strength

Build a Bayesian model for the 10th centile of log birth weight at each week, for both sexes, that

  • makes the curve smooth across weeks, with the amount of smoothing learned from the data;
  • shares information between boys and girls: a common curve plus a sex difference that is small and changes slowly;
  • uses the asymmetric Laplace working likelihood of E83 (pm.AsymmetricLaplace(..., q=0.1)).

Deliver

  1. Your priors, and why the smoothing prior is on the scale you chose.
  2. A fit with clean diagnostics.
  3. The chart with 90% bands on top of Task 1's points, and the boy-girl difference by week.
  4. The 10th centile at 40 weeks for girls and boys (posterior median, grams), and the held-out flag table of Task 1 for your new chart.
Your turn Work this task in the notebook.

Task 3 · Can you trust the bands?

The asymmetric Laplace is a working likelihood (E83). Find out what that does to this chart.

Deliver

  1. At the data-rich weeks (37-41), compare the posterior sd of each sex's 10th centile with an honest frequentist yardstick. Report the median ratio (posterior sd / yardstick sd).
  2. The held-out flag rates of Task 2 with an interval for each cell that includes both the binomial noise of the held-out cell and your posterior's uncertainty about the curve. How many cells fall outside their 90% interval? What can this check see, and what can it not?
  3. Fix the bands - one of E83's fixes is enough - refit, and redo 1 and 2. Did anything besides the band width change? (Look at the learned smoothness.)
Your turn Work this task in the notebook.

Task 4 · Charts for smokers and for gestational hypertension

The midwives' question needs conditional 10th centiles: the 10th centile of birth weight given week, sex, smoking and gestational hypertension.

Deliver

  1. A model for that, built on Task 3's (calibrated) chart. Think about whether the hypertension effect should be the same before and after 37 weeks - and about why it might not be.
  2. The effects at the 10th centile as percentages with 90% intervals, and a check that the intervals of the effect you will use in Task 5 are honest.
  3. Explain why the hypertension effect at the 10th centile, given gestational age, is so much smaller than the roughly -290 g E83 found at the 10th centile without gestational age in the model.
Your turn Work this task in the notebook.

Task 5 · The decision

Deliver, on the held-out births, comparing the conditional chart (Task 4) with the population chart (Task 3), with 90% intervals that include both charts' uncertainty and the sampling of the babies:

  1. Per 1000 babies of smokers, and per 1000 babies of hypertensive mothers: how many fewer are flagged under the conditional chart?
  2. Per 1000 births overall: how many babies stop being flagged, how many start being flagged, and the net change. Explain the sign of each.
  3. A recommendation to the obstetric lead in a few sentences - including a reason that does not come out of the posterior.
Your turn Work this task in the notebook.

Going further

  • A 3rd centile ("severe SGA") line. Fit it separately with the same model and check whether it crosses the 10th anywhere in the posterior draws; then fit both jointly with $\log q_{0.03} = \log q_{0.1} - e^{g}$ and a smooth $g$, which cannot cross. Does the 3rd centile need its own learning rate?
  • Does overconfidence roughen the curve? Refit both charts on a few other random training sets. Does the asymmetric Laplace sometimes learn a larger $s_f$ (a rougher curve) than the calibrated posterior? Why would an overconfident likelihood push the smoothing prior towards more curvature?
  • Measurement error in the week. Even the obstetric estimate is uncertain by about a week. Treat the true gestational age as latent (a mixture over neighbouring weeks) and see how much the preterm 10th centiles move.
  • A mixture for the lower tail. The 10th centile at 37 weeks is partly a statement about which babies are delivered early because they are small. Model birth weight at each week as a mixture of "normally grown" and "growth-restricted" components, and compare the flag it implies with the 10th centile.
  • Customised charts done right. Add maternal height and parity (physiological), not smoking or hypertension (pathological), and redo Task 5. Which babies change status now?
h.progress()

C14: 0/18 hints used, 10/10 checks passed.

Reference solution

A complete, executed solution with commentary on every task — including the ways the obvious model fails. Try the hint ladder first.

Open the worked solution →