07 · A scikit-learn workflow: build it, save it, share it

You will learn

• Read an input contract off a training frame
• Save and load a workflow bundle, and score new weeks
• See the labels when new data does not fit
• Publish, approve and serve a bundle (section 3)

You need

• Python 3.12 and uv
• Notebook 06 helps but is not required
• Section 3 is recorded; rerunning it needs the reference deployment

Run this notebook Offline, section 3 recorded

notebooks/07_workflow_sklearn.ipynb

Runs offline. Section 3 shows a recorded session against the reference deployment and cannot be run without it.

About 30 minutes

Access needed. The packages and notebooks are not yet publicly distributed. The steps below need a checkout of the UMP repository. Pages show recorded outputs, so you can follow along without running them.

cd notebooks && uv run jupyter lab

Runs offline; section 3 shows a recorded session against the reference deployment. Uses babelfin and scikit-learn. The data is SYNTHETIC, generated below with the same known truth as notebook 06.

Notebook 06 ended with Priya holding a good model and a run_workflow function that repeats it. Then the quarter turned. Twelve new weeks arrived, Sam (who approves what goes to the media lead) asked to rerun it himself, and Li on the planning team wanted the same answers inside her own notebook. A function in Priya’s notebook helps none of them: it lives in her kernel, the fitted pipeline lives in her memory, and nobody else can say which code produced which number.

This notebook turns notebook 06’s model into a workflow bundle: a folder holding the model’s source, its fitted state as plain numbers, an input contract and the named questions, every file hashed. Three steps:

1. Build: the model goes into a module, the questions get names and estimands, and the fit is checked against the truth.
2. Save and use: the bundle reruns on new weeks (score by default) and labels every time it cannot.
3. Share and serve: Priya publishes it, Sam approves it, Li runs it with no manual trust, and the bundle answers over HTTP as an app.

The next notebooks in the series build the same workflow with plain Python functions and with JAX, along the same three steps, so the toolkits can be compared cell by cell.

In [1]:

import os
import shutil
import socket
import subprocess
import sys
import tempfile
import time
from pathlib import Path

import babelfin as fin
import finprint as fp
import httpx
import numpy as np
import pandas as pd
from ump_sdk.ports import base_url

OUT = Path("out/workflows")
OUT.mkdir(parents=True, exist_ok=True)
CHANNELS = ["tv", "search", "social"]
SEED = 11


pd.set_option("display.max_colwidth", None)   # an estimand is a sentence; never cut it short


def table(numbers):
    """fin.Numbers as rows: value, interval and the estimand in words."""
    return pd.DataFrame([{"question": n, "value": a.value, "lo": a.lo, "hi": a.hi, "estimand": a.estimand.words}
                         for n, a in numbers.items()]).round(2)

1. Build

A synthetic world with a known truth

The same three years as notebook 06 (same seed, same adstock and saturation), without the duplicated and missing weeks: notebook 06 showed the prep, and here the point is the model. On top come the next quarter’s twelve weeks, whose spend reuses levels the brand has run before. Sales keep their carryover across the join.

In [2]:

TRUE = {"decay": {"tv": 0.5, "search": 0.0, "social": 0.3}, "slope": {"tv": 1.8, "search": 3.0, "social": 2.4}}


def adstock(x, decay):
    out = np.zeros(len(x))
    for t, v in enumerate(x):
        out[t] = v + (decay * out[t - 1] if t else 0.0)
    return out


def saturate(a, k):
    return k * np.log1p(a / k)


def make_world(seed=SEED, weeks=156, new_weeks=12):
    rng, later = np.random.default_rng(seed), np.random.default_rng(seed + 1)
    spend = pd.DataFrame({"tv": rng.gamma(4, 9000, weeks), "search": rng.gamma(5, 2500, weeks),
                          "social": rng.gamma(3, 4000, weeks)}).round(2)
    noise = rng.normal(0, 8_000, weeks)
    spend = pd.concat([spend, spend.apply(lambda s: later.choice(s.to_numpy(), new_weeks))], ignore_index=True)
    noise = np.concatenate([noise, later.normal(0, 8_000, new_weeks)])
    n = weeks + new_weeks
    t, promo = np.arange(n), (np.arange(n) % 9 == 3).astype(int)
    ads = {c: adstock(spend[c].to_numpy(), TRUE["decay"][c]) for c in CHANNELS}
    k = {c: ads[c][:weeks].mean() for c in CHANNELS}                  # half-saturation set on the three years

    def contrib(c, a):
        return TRUE["slope"][c] * saturate(a, k[c])

    sales = 200_000 + 30_000 * np.sin(2 * np.pi * t / 52) + 15_000 * promo + sum(contrib(c, ads[c]) for c in CHANNELS)
    frame = pd.DataFrame({"week": pd.date_range("2023-01-02", periods=n, freq="W-MON"),
                          "sales": (sales + noise).round(2), **spend, "promo": promo})
    window = slice(weeks, n)
    truth = {"three years": {c: contrib(c, ads[c])[:weeks].sum() / spend[c][:weeks].sum() for c in CHANNELS},
             "next quarter": {}}
    for c in CHANNELS:                                                 # remove the channel in the new weeks only
        cut = spend[c].to_numpy().copy()
        cut[window] = 0.0
        lost = contrib(c, ads[c])[window] - contrib(c, adstock(cut, TRUE["decay"][c]))[window]
        truth["next quarter"][c] = lost.sum() / spend[c][window].sum()
    return frame.iloc[:weeks].reset_index(drop=True), frame.iloc[weeks:].reset_index(drop=True), truth


train, new, TRUTH = make_world()
print(f"training weeks: {len(train)} ({train.week.min():%Y-%m-%d} to {train.week.max():%Y-%m-%d}); "
      f"new weeks: {len(new)} ({new.week.min():%Y-%m-%d} to {new.week.max():%Y-%m-%d})")
train.head(3)

Out [2]:

training weeks: 156 (2023-01-02 to 2025-12-22); new weeks: 12 (2025-12-29 to 2026-03-16)

	week	sales	tv	search	social	promo
0	2023-01-02	289765.35	33592.78	4362.40	11666.25	0
1	2023-01-09	357400.09	58926.07	14329.35	9511.26	0
2	2023-01-16	379233.63	28126.64	22913.72	25251.23	0

The input contract

What “the same necessary shape” means for this workflow, read off the training frame: these columns with these types, weekly on Mondays, sales as the target. More rows are always fine; a column it needs is not optional.

In [3]:

contract = fin.Contract.from_frame(train, time="week", target="sales", freq="W-MON")
contract

Out [3]:

Contract: 4 required, 0 optional columns, time week, target sales

ump.input_contract/0.1

details

key	week
extra columns	drop
categories	none

column	dtype	required	missing
tv	float64	True	forbid
search	float64	True	forbid
social	float64	True	forbid
promo	int64	True	forbid

The model, in a module

A bundle saves source code, never live objects, so the model has to live in a file the bundle can copy and hash: a function defined in a notebook cell exists only in this kernel. The module below is notebook 06’s model nearly line for line: the MediaTransform and Seasonality transformers in a ColumnTransformer, a LinearRegression per bootstrap draw, and each draw re-tuning the decays and saturation on the weeks it left out.

Four kinds of function make it a workflow: fit(df, *, seed, params), to_state and from_state, and one function per question that takes the fitted model and a frame and returns one value per draw. Questions work on the frame they are given, so the same roi_tv answers for three years at fit time and for twelve weeks at score time.

Why to_state exists: the fitted scikit-learn objects are not saved, because saving them would mean pickling, and unpickling a file runs whatever code it names. So to_state writes plain numbers (each draw’s coefficients, the tuned decays and saturation points, the first training week and the last 52 weeks of spend for carryover), and from_state rebuilds the transformers and regressions from them.

In [4]:

%%writefile out/workflows/northwind_sklearn.py
"""Northwind weekly sales: notebook 06's scikit-learn model as a workflow module (SYNTHETIC data).

fit() tunes and bootstraps the pipeline; to_state()/from_state() keep plain arrays only; one function per question.
"""
import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import ParameterGrid

CHANNELS = ["tv", "search", "social"]
GRID = {"tv_decay": [0.3, 0.5, 0.7], "social_decay": [0.0, 0.3, 0.5], "search_decay": [0.0, 0.3],
        "k_mult": [0.5, 1.0, 2.0]}
TAIL = 52   # weeks of spend kept for carryover into the next weeks (0.7 ** 52 is about 1e-8)


def adstock(x, decay):
    out = np.zeros(len(x))
    for t, v in enumerate(x):
        out[t] = v + (decay * out[t - 1] if t else 0.0)
    return out


def saturate(a, k):
    return k * np.log1p(a / k)


class MediaTransform(BaseEstimator, TransformerMixin):
    """Adstock then saturation per channel; keeps the spend it saw so later weeks start with their carryover."""

    def __init__(self, tv_decay=0.5, search_decay=0.0, social_decay=0.3, k_mult=1.0):
        self.tv_decay, self.search_decay, self.social_decay, self.k_mult = tv_decay, search_decay, social_decay, k_mult

    def _adstocked(self, X):
        known = getattr(self, "history_", None)
        earlier = known[known["week"] < X["week"].min()] if known is not None else X.iloc[:0]
        full = pd.concat([earlier, X[["week", *CHANNELS]]], ignore_index=True)
        n = len(earlier)
        return {c: adstock(full[c].to_numpy(dtype=float), getattr(self, f"{c}_decay"))[n:] for c in CHANNELS}

    def fit(self, X, y=None):
        self.history_ = None
        self.k_ = {c: self.k_mult * a.mean() for c, a in self._adstocked(X).items()}
        self.history_ = X[["week", *CHANNELS]].copy()
        return self

    def transform(self, X):
        return np.column_stack([saturate(a, self.k_[c]) for c, a in self._adstocked(X).items()])


class Seasonality(BaseEstimator, TransformerMixin):
    """A yearly cycle and a trend, counted in weeks from the first training week."""

    def fit(self, X, y=None):
        self.start_ = pd.to_datetime(X["week"]).min()
        return self

    def transform(self, X):
        t = (pd.to_datetime(X["week"]) - self.start_).dt.days.to_numpy() / 7.0
        return np.column_stack([np.sin(2 * np.pi * t / 52), np.cos(2 * np.pi * t / 52), t / 52])


def make_features(**media):
    return ColumnTransformer([("media", MediaTransform(**media), ["week", *CHANNELS]),
                              ("season", Seasonality(), ["week"]), ("promo", "passthrough", ["promo"])])


class Model:
    """The fitted workflow: one media step per tuned grid point, one seasonality step, a regression per draw."""

    def __init__(self, media, season, draws):
        self.media, self.season, self.draws = media, season, draws   # draws: [(grid point, LinearRegression)]

    def sales(self, df, **spend):
        """Weekly sales per draw (draws x weeks) on df's weeks, some channels' spend replaced."""
        X = df.assign(**spend)
        other = np.column_stack([self.season.transform(X), X["promo"].to_numpy(dtype=float)])
        F = {i: np.column_stack([m.transform(X), other]) for i, m in self.media.items()}
        return np.stack([reg.predict(F[i]) for i, reg in self.draws])


def fit(df, *, seed, params):
    """Notebook 06's block bootstrap: each draw resamples 8-week blocks, picks its own decays and saturation on
    the weeks it left out, and refits the regression."""
    points = list(ParameterGrid(GRID))
    feats = [make_features(**p).fit(df) for p in points]
    F = [f.transform(df) for f in feats]
    y, n, block = df["sales"].to_numpy(dtype=float), len(df), int(params.get("block", 8))
    rng = np.random.default_rng(seed)
    starts = np.arange(max(n - block + 1, 1))
    draws = []
    for _ in range(int(params.get("draws", 300))):
        rows = np.concatenate([np.arange(s, s + block) for s in rng.choice(starts, n // block + 1)])[:n]
        oob = np.setdiff1d(np.arange(n), rows)
        oob = oob if len(oob) else rows
        fits = [LinearRegression().fit(Fi[rows], y[rows]) for Fi in F]
        errs = [np.mean(np.abs(m.predict(Fi[oob]) - y[oob]) / y[oob]) for m, Fi in zip(fits, F)]
        best = int(np.argmin(errs))
        draws.append((best, fits[best]))
    media = {i: feats[i].named_transformers_["media"] for i in sorted({i for i, _ in draws})}
    return Model(media, feats[0].named_transformers_["season"], draws)


def to_state(model):
    """Plain numbers only: nothing is pickled, so the sklearn objects are rebuilt from these on load."""
    used = sorted(model.media)
    any_media = model.media[used[0]]
    tail = any_media.history_.tail(TAIL)
    meta = {"channels": CHANNELS, "start": model.season.start_.strftime("%Y-%m-%d"),
            "tail_weeks": pd.to_datetime(tail["week"]).dt.strftime("%Y-%m-%d").tolist()}
    arrays = {"point": np.array([used.index(i) for i, _ in model.draws]),
              "decay": np.array([[getattr(model.media[i], f"{c}_decay") for c in CHANNELS] for i in used]),
              "k_mult": np.array([model.media[i].k_mult for i in used]),
              "k": np.array([[model.media[i].k_[c] for c in CHANNELS] for i in used]),
              "coef": np.stack([reg.coef_ for _, reg in model.draws]),
              "intercept": np.array([reg.intercept_ for _, reg in model.draws]),
              "tail": tail[CHANNELS].to_numpy(dtype=float)}
    return meta, arrays


def from_state(meta, arrays):
    history = pd.DataFrame(np.asarray(arrays["tail"]), columns=meta["channels"])
    history.insert(0, "week", pd.to_datetime(meta["tail_weeks"]))
    media = {}
    for i, (decay, k_mult, k) in enumerate(zip(arrays["decay"], arrays["k_mult"], arrays["k"])):
        m = MediaTransform(**{f"{c}_decay": float(d) for c, d in zip(CHANNELS, decay)}, k_mult=float(k_mult))
        m.k_, m.history_ = dict(zip(CHANNELS, map(float, k))), history
        media[i] = m
    season = Seasonality()
    season.start_ = pd.Timestamp(meta["start"])
    draws = []
    for i, coef, icept in zip(arrays["point"], arrays["coef"], arrays["intercept"]):
        reg = LinearRegression()
        reg.coef_, reg.intercept_, reg.n_features_in_ = np.asarray(coef), float(icept), len(coef)
        draws.append((int(i), reg))
    return Model(media, season, draws)


# ---- the named questions: each returns one value per draw, over the weeks of the data it is given

def _roi(m, df, ch):
    return (m.sales(df) - m.sales(df, **{ch: 0.0})).sum(axis=1) / df[ch].sum()


def _mroi(m, df, ch):
    return (m.sales(df, **{ch: df[ch] * 1.1}) - m.sales(df)).sum(axis=1) / (0.1 * df[ch].sum())


def roi_tv(m, df):
    return _roi(m, df, "tv")


def roi_search(m, df):
    return _roi(m, df, "search")


def roi_social(m, df):
    return _roi(m, df, "social")


def mroi_tv(m, df):
    return _mroi(m, df, "tv")


def mroi_search(m, df):
    return _mroi(m, df, "search")


def mroi_social(m, df):
    return _mroi(m, df, "social")


def cut_tv_20pct(m, df):
    return (m.sales(df, tv=df["tv"] * 0.8) - m.sales(df)).sum(axis=1)

Out [4]:

Overwriting out/workflows/northwind_sklearn.py

Named questions

Each question names a function in the module, says in words what it asks, and carries the estimand that pins down what its number means. The average ROIs and next-dollar mROIs also name their row in the bundle’s summary table, so a reader who cannot run the code still gets a labelled number for them (more on that in section 2).

In [5]:

COMMON = {"kpi": "incremental_sales", "cost_basis": "gross", "point_rule": "posterior_median",
          "interval": "bootstrap_pct"}

wf = fin.Workflow("Northwind sklearn MMM (synthetic data)", contract=contract,
                  code=OUT / "northwind_sklearn.py", seed=SEED, params={"draws": 300}, dependence="joint_draws")
for ch in CHANNELS:
    wf.question(f"roi_{ch}", ask=f"What did each dollar of {ch} return over these weeks?",
                estimand=fin.estimand("roi", channels=ch, **COMMON), summary=("roi", ch))
    wf.question(f"mroi_{ch}", ask=f"What would the next dollar of {ch} return?",
                estimand=fin.estimand("mroi", channels=ch, **COMMON), summary=("mroi", ch))
wf.question("cut_tv_20pct", ask="What happens to sales if we cut TV by 20%?",
            estimand=fin.estimand("roi", channels="tv", intervention="scale", size=-0.2, baseline="current_plan",
                                  per="absolute", **COMMON))
wf

Out [5]:

Northwind sklearn MMM (synthetic data)

workflow, not fitted · 7 questions

details

module	northwind_sklearn.py
seed	11
prep recipe	none
contract	<Contract: 4 required, 0 optional columns, time week, target sales>

question	ask
roi_tv	What did each dollar of tv return over these weeks?
mroi_tv	What would the next dollar of tv return?
roi_search	What did each dollar of search return over these weeks?
mroi_search	What would the next dollar of search return?
roi_social	What did each dollar of social return over these weeks?
mroi_social	What would the next dollar of social return?
cut_tv_20pct	What happens to sales if we cut TV by 20%?

Fit and look at the answers

fit checks the frame against the contract, runs the module’s fit and answers every question on the training weeks. Each answer is a fin.Number: the median of the draws with a 90% bootstrap interval, carrying its estimand. The horizon is the weeks of the frame (156 here), which is why each number’s estimand says so.

In [6]:

fitted = wf.fit(train)
answers = fitted.answers()
table(answers)

Out [6]:

	question	value	lo	hi	estimand
0	roi_tv	2.22	1.86	2.69	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 156 weeks, posterior median, 90% bootstrap_pct
1	mroi_tv	1.53	1.15	1.65	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 156 weeks, posterior median, 90% bootstrap_pct
2	roi_search	1.70	1.36	2.30	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 156 weeks, posterior median, 90% bootstrap_pct
3	mroi_search	1.11	0.90	1.41	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 156 weeks, posterior median, 90% bootstrap_pct
4	roi_social	2.59	2.02	3.00	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 156 weeks, posterior median, 90% bootstrap_pct
5	mroi_social	1.65	1.50	1.81	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 156 weeks, posterior median, 90% bootstrap_pct
6	cut_tv_20pct	-1910074.61	-2046383.37	-1445858.93	incremental sales in total, gross from scaling spend (-20%), geo_set, versus the current plan, over 156 weeks, posterior median, 90% bootstrap_pct

Did the model get it right?

The world is synthetic, so the true average ROI over the three years is known (the same truth notebook 06 checked). Each 90% interval should cover it.

In [7]:

check = pd.DataFrame([{"channel": c, "estimate": answers[f"roi_{c}"].value, "lo": answers[f"roi_{c}"].lo,
                       "hi": answers[f"roi_{c}"].hi, "truth": TRUTH["three years"][c]} for c in CHANNELS])
check["covered"] = (check.lo <= check.truth) & (check.truth <= check.hi)
check.round(3)

Out [7]:

	channel	estimate	lo	hi	truth	covered
0	tv	2.218	1.859	2.690	2.457	True
1	search	1.698	1.358	2.304	2.014	True
2	social	2.586	2.024	3.000	2.290	True

2. Save and use

Save

save writes the bundle. Before it writes anything it rebuilds the model with from_state(to_state(model)) and checks the rebuilt one gives the same draws on the training weeks; a state that loses information is refused and nothing is written. Training data stays out unless Priya opts in with include_data=True.

In [8]:

BUNDLE = OUT / "northwind_sklearn.bundle"
shutil.rmtree(BUNDLE, ignore_errors=True)          # save needs a fresh folder; this notebook rewrites its own
path = fitted.save(BUNDLE, author="user:priya.shah")
print(path.name)
print("\n".join(sorted(p.relative_to(path).as_posix() for p in path.rglob("*") if p.is_file())))

Out [8]:

northwind_sklearn.bundle
README.md
SHA256SUMS
code/northwind_sklearn.py
contract.json
questions.json
state/coef.npy
state/decay.npy
state/intercept.npy
state/k.npy
state/k_mult.npy
state/point.npy
state/state.json
state/tail.npy
summary.json
workflow.json

On a new machine

Sam copies the folder to his laptop. The next cell stands in for that laptop: an empty trust store, and the hash Priya’s own save trusted for this session forgotten. On a real second machine there is nothing to forget.

In [9]:

os.environ["FINPRINT_HOME"] = tempfile.mkdtemp(prefix="sam-laptop-")
fp.trust._SESSION.clear()

Loading checks every file’s hash and parses the data files. It runs nothing: Sam’s laptop has never seen this code.

In [10]:

lw = fin.Workflow.load(BUNDLE)
print("trusted:", lw.trusted)
print("code sha256:", lw.code_sha256[:16], "...")
print("\n".join(lw.code.splitlines()[:12]))

Out [10]:

trusted: False
code sha256: dfba6cbbb408fe76 ...
"""Northwind weekly sales: notebook 06's scikit-learn model as a workflow module (SYNTHETIC data).

fit() tunes and bootstraps the pipeline; to_state()/from_state() keep plain arrays only; one function per question.
"""
import numpy as np
import pandas as pd
from sklearn.base import BaseEstimator, TransformerMixin
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import ParameterGrid

CHANNELS = ["tv", "search", "social"]

Until Sam trusts the code, asking a question returns the bundle’s summary: the three-year numbers saved at fit time, labelled as a fallback, at trust level 1. They answer a different window than the twelve weeks he asked about, and the labels say so. The cut-TV question has no summary row, so it is left out rather than guessed.

In [11]:

untrusted = lw.score(new)
print(untrusted.route, "/", untrusted.source, "|", ", ".join(n["code"] for n in untrusted.notes))
table(untrusted.answers())

Out [11]:

not_available / summary | WORKFLOW_CODE_UNTRUSTED, SUMMARY_FALLBACK

	question	value	lo	hi	estimand
0	roi_tv	2.22	1.86	2.69	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 156 weeks, posterior median, 90% bootstrap_pct
1	mroi_tv	1.53	1.15	1.65	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 156 weeks, posterior median, 90% bootstrap_pct
2	roi_search	1.70	1.36	2.30	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 156 weeks, posterior median, 90% bootstrap_pct
3	mroi_search	1.11	0.90	1.41	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 156 weeks, posterior median, 90% bootstrap_pct
4	roi_social	2.59	2.02	3.00	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 156 weeks, posterior median, 90% bootstrap_pct
5	mroi_social	1.65	1.50	1.81	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 156 weeks, posterior median, 90% bootstrap_pct

Sam reads the code (all of lw.code, not just the first lines above), then trusts it. The trust is the code’s hash in his laptop’s store, so a changed file would not be trusted.

In [12]:

lw.trust()
lw.trusted

Out [12]:

True

Score twelve new weeks

The twelve weeks start right after the training weeks, so the saved fit scores them (score by default): no refit, the same draws, carryover from the last weeks of training spend. Each answer’s horizon is now 12 weeks.

In [13]:

run = lw.score(new)
print(run.route, "/", run.source, "/", run.rows, "rows")
table(run.answers())

Out [13]:

score / score / 12 rows

	question	value	lo	hi	estimand
0	roi_tv	2.03	1.74	2.46	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 12 weeks, posterior median, 90% bootstrap_pct
1	mroi_tv	1.48	1.18	1.58	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 12 weeks, posterior median, 90% bootstrap_pct
2	roi_search	1.70	1.36	2.22	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 12 weeks, posterior median, 90% bootstrap_pct
3	mroi_search	1.13	0.92	1.42	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 12 weeks, posterior median, 90% bootstrap_pct
4	roi_social	2.65	2.07	3.11	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 12 weeks, posterior median, 90% bootstrap_pct
5	mroi_social	1.78	1.61	1.95	incremental sales per dollar, gross from a small increase in spend (+10%), geo_set, versus the current plan, over 12 weeks, posterior median, 90% bootstrap_pct
6	cut_tv_20pct	-124037.27	-134379.83	-100938.16	incremental sales in total, gross from scaling spend (-20%), geo_set, versus the current plan, over 12 weeks, posterior median, 90% bootstrap_pct

Twelve weeks is a short window, so these intervals are the model’s uncertainty about one quarter. The truth for the window is known here too (removing each channel’s spend in the new weeks only):

In [14]:

scored = run.answers()
window = pd.DataFrame([{"channel": c, "estimate": scored[f"roi_{c}"].value, "lo": scored[f"roi_{c}"].lo,
                        "hi": scored[f"roi_{c}"].hi, "truth": TRUTH["next quarter"][c]} for c in CHANNELS])
window["covered"] = (window.lo <= window.truth) & (window.truth <= window.hi)
window.round(3)

Out [14]:

	channel	estimate	lo	hi	truth	covered
0	tv	2.035	1.737	2.458	2.235	True
1	search	1.702	1.359	2.225	2.015	True
2	social	2.655	2.068	3.108	2.346	True

When the new data does not fit

A gap. Priya’s next file appends the new weeks to the full history, and the feed dropped two of them. Scoring needs weeks that follow on from the saved fit without a break, and this file both repeats the old weeks and skips two new ones. So the run refits on the data it was given, with the saved seed and settings, and says so on every number (open the card’s details to see why).

In [15]:

gappy = pd.concat([train, new.drop(index=[3, 4])], ignore_index=True)
refit = lw.score(gappy)
print(refit.route, "/", refit.source)
print("\n".join(f"{n['code']}: {n['message']}" for n in refit.notes))

Out [15]:

refit / refit
INPUT_IRREGULAR_TIME: time column week is irregular (2 missing steps); scoring needs regular steps, so the rerun refits
INPUT_HISTORY_OVERLAP: new input starts 2023-01-02, on or before the history end 2025-12-22; the rerun refits
HISTORY_ROUTED_TO_REFIT: the input does not follow on from the saved history, so the saved state is not scored; the rerun refits
INPUT_ROWS_GREW: 166 rows, 156 at training

In [16]:

refit.answer("roi_tv")

Out [16]:

roi_tv: 2.30

likely between 1.75 and 2.78 · medium confidence

details

means	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 166 weeks, posterior median, 90% bootstrap_pct
interval	90% bootstrap_pct 1.75 to 2.78
trust level	2
G-DATA-07 INPUT_IRREGULAR_TIME	time column week is irregular (2 missing steps); scoring needs regular steps, so the rerun refits; answer refitted on this input with the saved seed and params, not the saved state
G-DATA-07 INPUT_HISTORY_OVERLAP	new input starts 2023-01-02, on or before the history end 2025-12-22; the rerun refits; answer refitted on this input with the saved seed and params, not the saved state
G-DATA-07 HISTORY_ROUTED_TO_REFIT	the input does not follow on from the saved history, so the saved state is not scored; the rerun refits; answer refitted on this input with the saved seed and params, not the saved state

An extra column. The new file carries a weather index the model never used. It is dropped with an info note, and the run still scores.

In [17]:

extra = lw.score(new.assign(weather_index=np.linspace(0, 1, len(new))))
print(extra.route, "/", extra.source)
print("\n".join(f"{n['code']}: {n['message']}" for n in extra.notes))

Out [17]:

score / score
INPUT_EXTRA_DROPPED: column weather_index is not used by the workflow; dropped

A missing column. This file lost its promo flag. The model cannot run without it, so the answers fall back to the bundle’s summary, labelled, and the cut-TV question is not available.

In [18]:

missing = lw.score(new.drop(columns="promo"))
print(missing.route, "/", missing.source)
print("\n".join(f"{n['code']}: {n['message']}" for n in missing.notes))

Out [18]:

not_available / summary
INPUT_REQUIRED_MISSING: required column promo is missing
SUMMARY_FALLBACK: answers are the bundle's fit-time Tier D summary because the input does not meet the contract; no summary number for cut_tv_20pct (not available)

In [19]:

missing.answer("roi_tv")

Out [19]:

roi_tv: 2.22

likely between 1.86 and 2.69 · low confidence

details

means	incremental sales per dollar, gross from removing all spend, geo_set, versus zero spend, over 156 weeks, posterior median, 90% bootstrap_pct
interval	90% bootstrap_pct 1.86 to 2.69
trust level	1
G-DATA-07 INPUT_REQUIRED_MISSING	required column promo is missing; answer is the bundle's fit-time Tier D summary, not a run on this input
G-DATA-07 SUMMARY_FALLBACK	answers are the bundle's fit-time Tier D summary because the input does not meet the contract; no summary number for cut_tv_20pct (not available); answer is the bundle's fit-time Tier D summary, not a run on this input

What is in the bundle

inspect() reads the folder without running anything: the contract, the questions, the toolkit versions the code imports (scikit-learn among them), and the fit-time summary.

In [20]:

info = lw.inspect()
{k: info[k] for k in ["name", "code_sha256", "env_pin", "questions"] if k in info}

Out [20]:

{'name': 'Northwind sklearn MMM (synthetic data)',
 'code_sha256': 'dfba6cbbb408fe762bb25278a0bcb10bcb853bcc4e2ddeb645573ae4dccf47de',
 'env_pin': {'python': '3.12.8',
  'numpy': '2.5.4',
  'pandas': '3.1.0',
  'finprint': '0.2.0',
  'ump_spec': '0.1.0',
  'sklearn': '1.9.1'},
 'questions': [{'name': 'roi_tv',
   'ask': 'What did each dollar of tv return over these weeks?'},
  {'name': 'mroi_tv', 'ask': 'What would the next dollar of tv return?'},
  {'name': 'roi_search',
   'ask': 'What did each dollar of search return over these weeks?'},
  {'name': 'mroi_search',
   'ask': 'What would the next dollar of search return?'},
  {'name': 'roi_social',
   'ask': 'What did each dollar of social return over these weeks?'},
  {'name': 'mroi_social',
   'ask': 'What would the next dollar of social return?'},
  {'name': 'cut_tv_20pct',
   'ask': 'What happens to sales if we cut TV by 20%?'}]}

3. Share and serve (recorded session)

Copying folders by hand does not scale past Sam. Through UMP the bundle is filed as a Tier A model whose files are stored by hash; a second person approves it; after that, colleagues and servers run it without each trusting it by hand. Everything here goes to northwind-e2e-live, the labelled test client, never to the demo client.

Recorded run

This section shows a recorded session against the reference deployment. It cannot be run without that deployment; the offline sections above run anywhere. The identity stub (as_=) and the localhost address belong to the reference deployment, not to the standard; a conforming core authenticates callers as section 19 requires.

In [21]:

try:
    CORE_UP = httpx.get(base_url("core") + "/healthz", timeout=2).status_code == 200
except httpx.HTTPError:
    CORE_UP = False
CLIENT = "northwind-e2e-live"
NEEDS_POC = "This section needs the running POC (scripts/dev.sh); skipped."
print("core is up" if CORE_UP else NEEDS_POC)

Out [21]:

core is up

Publish and submit

In the POC, Priya’s team files its work under Maya Chen’s analyst account, so Maya publishes. publish saves the bundle with her as author, uploads every file (by content hash) and files a draft Tier A model manifest; submit sends it for approval.

In [22]:

if CORE_UP:
    maya = fin.connect(as_="maya.chen")
    art = maya.workflows.publish(fitted, client=CLIENT, title="Northwind sklearn workflow (synthetic, notebook 07)")
    art.submit(reason="notebook 07: shared scikit-learn workflow")
    print(art.id, art.state, "| tier", art.doc["tier"], "| engine", art.doc["engine"])
else:
    print(NEEDS_POC)

Out [22]:

mdl_72b401e75a4648c681a2 awaiting_approval | tier A | engine ump-workflow

Maya cannot approve her own work (G-INT-02, one of the hard stops). Core refuses it.

In [23]:

if CORE_UP:
    try:
        art.approve(accept=art.fallbacks)
    except fin.Forbidden as exc:
        print("refused:", exc)
else:
    print(NEEDS_POC)

Out [23]:

refused: SELF_APPROVAL_FORBIDDEN: an approver cannot approve their own work (owner or submitter)

Sam approves after reading the code. An approval names every fallback it accepts, so he lists them first. Here there is one: the workflow has not been through the acceptance battery, so it is untested, which means usable at reduced trust rather than failed.

In [24]:

if CORE_UP:
    with maya.as_("sam.patel") as sam:
        pending = sam.artifact(art.id)
        print("fallbacks Sam accepts:", list(pending.fallbacks.keys))
        pending.approve(accept=pending.fallbacks, comment="read northwind_sklearn.py; accepts the listed fallback")
    print(art.refresh().state)
else:
    print(NEEDS_POC)

Out [24]:

fallbacks Sam accepts: ['G-ACC-01:UNVALIDATED_METHOD']
validated

A colleague runs it

Li Wong has never trusted this code and her trust store is empty (the next cell gives this kernel a fresh one). get fetches every file, checks its hash, rebuilds the folder and trusts exactly the hashes Sam’s approval vouches for. Nothing is written to Li’s store.

In [25]:

if CORE_UP:
    os.environ["FINPRINT_HOME"] = tempfile.mkdtemp(prefix="li-laptop-")
    fp.trust._SESSION.clear()
    li = fin.connect(as_="li.wong")
    li_copy = Path(tempfile.mkdtemp(prefix="li-")) / "northwind_sklearn"
    shared = li.workflows.get(art.id, dest=li_copy)
    print("trusted:", shared.trusted, "| reason:", shared.approval.reason)
    li_run = shared.score(new)
    print(li_run.route, "/", li_run.source)
    print(li_run.answer("roi_tv"))
else:
    print(NEEDS_POC)

Out [25]:

trusted: True | reason: None
score / score
<Number roi_tv: 2.03 (90% 1.74 to 2.46)>

Serve it as an app

ump-workflow serve puts one bundle behind UMP’s adapter contract. Here it fetches the approved manifest itself, with its own empty trust store, so the server runs the code only because Sam approved it. Li asks it the same questions over HTTP and gets the same kind of fin.Numbers back.

In [26]:

if CORE_UP:
    with socket.socket() as sock:
        sock.bind(("127.0.0.1", 0))
        port = sock.getsockname()[1]
    server_dir = Path(tempfile.mkdtemp(prefix="served-"))
    server = subprocess.Popen(
        [sys.executable, "-m", "ump_workflow", "serve", "--artifact", art.id, "--principal", "li.wong",
         "--port", str(port), "--dest", str(server_dir / "bundle")],
        env={**os.environ, "FINPRINT_HOME": str(server_dir / "home")},
        stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
    url = f"http://127.0.0.1:{port}"
    for _ in range(200):
        try:
            if httpx.get(url + "/healthz", timeout=1).status_code == 200:
                break
        except httpx.HTTPError:
            time.sleep(0.3)
    served = li.workflows.answer(url, new)
    print(served.route, "/", served.source, "/", served.rows, "rows")
    print(served.answer("mroi_tv"))
else:
    print(NEEDS_POC)

Out [26]:

score / score / 12 rows
<Number mroi_tv: 1.48 (90% 1.18 to 1.58)>

Stop the server, and retire the published manifest. A validated workflow manifest shows up in the models app as a Tier A model without curves, so this notebook moves its own to deprecated rather than leave it there.

In [27]:

if CORE_UP:
    server.terminate()
    server.wait(timeout=15)
    maya.core.transition(art.id, "deprecated", reason="notebook 07 cleanup (synthetic shared workflow)")
    print("server stopped; manifest", art.refresh().state)
    li.close()
    maya.close()
else:
    print(NEEDS_POC)

Out [27]:

server stopped; manifest deprecated

What you have now

• Notebook 06’s scikit-learn model as a module, with seven named questions, each answer a fin.Number with its estimand and interval, checked against the truth.
• A bundle in out/workflows/ that reruns on new weeks: it scores when it can and labels every refit and fallback when it cannot.
• The same bundle shared through UMP: one approval by a second person, after which Li and a served app run it with no manual trust.

Next in the series: the same workflow with plain Python functions, then with JAX.