Challenges
Challenges
Eleven problems on real data, each ending in a decision someone actually has to make. In every one, the obvious first model breaks in a way you have to diagnose. The brief tells you what to deliver, never how.
How a challenge works
A brief and a set of tasks
Each challenge page describes the setting, the data and a sequence of tasks with concrete deliverables — a plot, a number, a recommendation. Download the notebook and work in the empty cells.
A ladder of hints
Every task has three hints, revealed one at a time: a nudge (where to look), an approach (the modelling idea and the API) and a skeleton (code with the key parts blank). A good target is to finish on nudges alone.
Checks and a solution
Numeric self-checks with generous ranges tell you whether your model is in the right place. A reference solution, fully executed and discussed, is there for when you have finished — or are truly stuck.
They are roughly in order of difficulty but independent, so pick the one closest to your own work. The descriptions are deliberately vague about what goes wrong: finding out is the challenge.
The challenges
| Challenge | Difficulty | You will have to | |
|---|---|---|---|
| C01 | Golf putting: physics beats curve fitting Berry (1996) + Broadie PGA putts | ★★★★★ | build a likelihood from geometry; work out why a perfectly sensible model will not mix on the bigger dataset |
| C02 | Premier League forecasting football-data.co.uk 2024-26 | ★★★★★ | rescue a team-strength model whose first version samples badly; simulate the rest of a season; score yourself against the bookmakers |
| C03 | Bike-share demand Capital Bikeshare daily counts | ★★★★★ | catch a model with perfect diagnostics and useless intervals; model non-linear weather effects; provision a fleet from a predictive quantile |
| C04 | Customer churn as survival IBM Telco churn | ★★★★★ | handle censoring; design a PPC for censored data; target a retention offer by expected value |
| C05 | Likert scales are not numbers Trolley-problem moral judgements | ★★★★★ | replace linear regression with an ordered-logistic model; interpret on the outcome scale |
| C06 | How many galaxy clusters? Corona Borealis velocities | ★★★★★ | explain why four healthy-looking chains disagree; choose the number of clusters; report co-clustering probabilities |
| C07 | Stochastic volatility and VaR S&P 500 daily returns | ★★★★★ | sample a latent volatility path without being fooled by a silent sampler failure; backtest a 99% Value-at-Risk |
| C08 | Forecasting atmospheric CO2 NOAA Mauna Loa record | ★★★★★ | build an additive GP; work out why a stationary GP cannot forecast a trend; tame non-identified components with priors; date a threshold crossing |
| C09 | Does job training work? LaLonde NSW experiment + CPS | ★★★★★ | g-computation with pm.do; find out why a confident, clean-sampling estimate is wrong; compare with the experimental benchmark |
| C10 | Who draws fouls in the NBA? Last Two Minute reports 2015-21 | ★★★★★ | fit an item-response model with ~1500 player effects; repair a colleague's broken model; measure (not assume) what reparameterising buys; rank with uncertainty |
| C11 | Who needs the second drug? ACTG 175 HIV trial (zidovudine vs zidovudine + didanosine) | ★★★★★ | build a Dirichlet-process mixture of regressions; take apart a colleague's convincing "responder" analysis; let covariates choose the subgroups and find out what LOO cannot tell you; turn conditional effects into a treatment rule |
| C12 | Crosswinds at Santa Barbara SBA airport hourly METARs 2021-23 (IEM) | ★★★★★ | turn wind reports into a solo-flight schedule; find out why two models that each pass their own checks combine into a confident wrong answer; check calibration on a held-out year; pick a default time window with uncertainty |
| C13 | Scale-up from a partial screen Shields et al. (2021) direct-arylation screen, 60 of 1728 reactions | ★★★★★ | model bounded yields from a small random screen; predict the whole grid and P(best); decide whether another round of experiments pays, then audit model and decision against the full table |
| C14 | Small for gestational age: whose chart flags whom? US natality 2023 (NCHS, 20k-birth sample) | ★★★★★ | choose a gestational age and clean without deleting the babies the chart is for; smooth a sparse preterm tail; find out whether your bands mean what they say; turn two charts into "babies per 1000 flagged" and a recommendation |
| C15 | Should Parkay run a promotion? Margarine scanner panel (Allenby & Rossi 1991, bayesm) | ★★★★★ | price a 20% promotion with a household-level choice model; find out what a pooled model quietly assumes about cannibalisation and loyalty; score models on held-out purchases; turn the posterior into a profit distribution and a go/no-go |
Working locally
In the notebook, a helper object tracks your hints and checks:
h.tasks() # the tasks, and how many hints / checks each has
h.hint("task2") # reveal the next hint for task 2
h.check("task2", sigma_angle_deg=1.5) # compare your number with the reference
h.progress() # hints used, checks passedSee About for setting up the environment.