Tabular models with scikit-learn and xgboost¶

mapi-sklearn changes each scikit-learn, xgboost, LightGBM or CatBoost estimator on the allow-list into a full modeling-api plugin. You tell which dims are samples and which variables are features. The platform does the rest:

It stacks your labelled xr.Dataset into a design matrix.
It fits the estimator and keeps it safely.
At score time, it builds exactly the same columns again.
It accepts new stores and forecast horizons.
It unstacks the predictions back into labelled arrays, with correct uncertainty.

The plugins¶

The kind of uncertainty sets the representation of the output. The representation then sets which summaries and aggregations are valid. Each kind is a different plugin:

Plugin	Representation	What `y_hat` contains	Aggregate over cells?
`sklearn.tabular_regressor`	point	A point prediction (+ `contributions`)	Linear only, with a flag
`sklearn.tabular_quantiles`	quantiles	Conditional quantiles `(quantile, ...)`	No
`sklearn.tabular_bootstrap`	draws	One joint draw for each bootstrap replicate `(chain, draw, ...)`	Yes, all
`sklearn.tabular_conformal`	interval	`(bound=[lower, upper, point], ...)`, attr `alpha`	No
`sklearn.tabular_classifier`	point	The probability of the positive class	Linear only, with a flag
`sklearn.multiclass_classifier` (v0.14)	point	The probability of each class: `y_hat(..., class)`	No

Why conformal is an interval, not draws

A split-conformal interval guarantees the coverage for each cell (P(y in interval) >= 1 - alpha). It is not a distribution. A sum of intervals across stores or weeks claims a coverage that nobody measured. Thus the platform rejects these aggregations with CAPABILITY_UNSUPPORTED. For totals, use sklearn.tabular_bootstrap, because its draws are joint.

A store-demand forecast¶

apiVersion: mapi/v1
kind: ModelSpec
metadata: {name: store-demand-xgb, project: retail}
data:
  dataset: dataset://retail/store_sku_weekly@latest
  bind:
    dims: {store: store_id, sku: sku, time: week}
    vars: {y: units_sold, price: price, promo: promo_flag}
  split: {kind: time_holdout, dim: time, last: 12}
pipeline:
  - {id: imp,   use: core.impute@^1,   with: {vars: [price], method: ffill, along: time, within: [store, sku]}}
  - {id: plags, use: core.lag@^1,      with: {vars: [promo], along: time, within: [store, sku], lags: [1, 2]}}
  - {id: cal,   use: core.calendar@^1, with: {along: time, features: [week_of_year, month]}}
  - {id: logy,  use: core.log1p@^1,    with: {vars: [y]}}
model:
  use: sklearn.tabular_quantiles@^1
  settings:
    estimator: {kind: xgboost.XGBRegressor, params: {n_estimators: 600, max_depth: 6, learning_rate: 0.05}}
    rows: [store, sku, time]                  # dims stacked into samples
    columns: {roles: [covariate]}             # which variables become features
    coord_features: {store: category, sku: category}
    uncertainty: {kind: quantile, quantiles: [0.1, 0.5, 0.9]}
    forecast_strategy: direct
  fit: {seed: 7, mode: evaluate_then_refit}
evaluate: {metrics: [rmse, mape, coverage_80]}
execution: {resource_class: S, timeout: 10m}

What occurs:

The platform scores the holdout (the last 12 weeks) through the score path. It uses context tails from the training weeks. It calculates rmse, mape and coverage_80 in units sold. The platform inverts the log1p. Quantiles map exactly through monotone transforms.
The platform fits the model again on all weeks. The bundle keeps the xgboost model as JSON.
A forecast request gives the future prices and promotions. The platform builds the lags again from the stored tail. It also does a check that the weeks continue the fitted range without gaps.

apiVersion: mapi/v1
kind: ScoreRequest
model: endpoint://retail/store-demand-xgb
mode: forecast
horizon: {dim: time, periods: 4}
data: dataset://retail/plan_next_4_weeks
output: {summary: [median, q10, q90]}

Settings¶

Setting	Default	Meaning
`estimator`	—	`{kind, params}` from the allow-list below (necessary for the built-in plugins)
`target`	`y`	The canonical name of the target variable
`rows`	—	The dims that the platform stacks into samples. Each row dim of the target.
`time_dim` / `time_kind`	`time` / `time`	The row dim that extends at forecast time, and its kind (`time`, `integer`, `index`). Use `none` if no row dim extends.
`columns`	`{roles: [covariate, media, spend, control, exposure]}`	The feature variables by role, or an explicit `vars` list. `exclude` removes names.
`coord_features`	`{}`	Encode the labels of row dims as features: `category`, `onehot`, `ordinal`, `numeric`
`new_levels`	`pooled`	What an unseen level of a row dim becomes: `pooled`, `zero` or `error`
`pooled_max_levels`	256	Pool over a maximum of this number of fitted levels (the most frequent)
`weight`, `mask`	—	A variable for the sample weight. A boolean mask of the cells that you can use.
`forecast_strategy`	`direct`	In v1, the platform rejects `recursive` with `RECURSIVE_UNSUPPORTED`.
`retransform`	`median`	Point predictions through a non-linear inverse (`log1p`): `median` or Duan `smearing`
`contributions`	`auto`	For the point and bootstrap plugins: `auto`, `exact`, `zeroing`, `none`
`n_jobs`	1	The threads of the estimator. The default is 1, so that fits are the same bit for bit.

A variable with dims that are not in rows becomes one column for each label. For example, spend(store, channel, time) with the rows [store, time] gives spend[tv], spend[search]. The platform broadcasts a variable that does not have a row dim (calendar features over time only). The platform does not use some rows for the training:

Rows with a missing target
Rows with a weight that is zero or less
Rows where mask is false
Rows with a missing feature, for estimators that cannot accept missing values natively

A ROWS_DROPPED diagnostic reports these rows.

Coordinate features¶

category: integer codes. The platform declares them natively to xgboost and to HistGradientBoostingRegressor (a maximum of 254 levels). For all other estimators, it uses one-hot columns.
onehot: one 0/1 column for each fitted level.
ordinal: the position of the level as a number (for ordered dims).
numeric: the label. Numbers stay as they are. Datetimes become the days since the first fitted label.

Estimators¶

Kind	Task	Quantiles	Missing values	Categories	Stored as
`sklearn.LinearRegression`	regression	—	—	one-hot	skops
`sklearn.Ridge`, `sklearn.Lasso`, `sklearn.ElasticNet`	regression	—	—	one-hot	skops
`sklearn.QuantileRegressor`	regression	one model for each level	—	one-hot	skops
`sklearn.RandomForestRegressor`	regression	—	—	one-hot	skops
`sklearn.GradientBoostingRegressor`	regression	one model for each level	—	one-hot	skops
`sklearn.HistGradientBoostingRegressor`	regression	one model for each level	native	native	skops
`xgboost.XGBRegressor`	regression	one native multi-quantile model	native	native	xgboost JSON
`sklearn.LogisticRegression`, `sklearn.RandomForestClassifier`, `xgboost.XGBClassifier`	binary and multiclass classification	—	xgboost only	xgboost only	skops / JSON
`sklearn.HistGradientBoostingClassifier` (v0.14)	binary and multiclass classification	—	native	native	skops
`lightgbm.LGBMClassifier` (v0.14)	binary and multiclass classification	—	native	one-hot	LightGBM text model
`catboost.CatBoostClassifier` (v0.14)	binary and multiclass classification	—	native	one-hot	CatBoost JSON model

LightGBM and CatBoost are in the extra mapi-sklearn[gbm]. The base-ml runner image has this extra. The other runner images do not have LightGBM or CatBoost. If the extra is not installed, a fit with these estimators stops with SETTINGS_INVALID. The message tells you to install mapi-sklearn[gbm]. The platform sets their output to quiet and their threads from n_jobs. CatBoost writes no files outside the run.

Each kind has a bounded Pydantic params model (extra="forbid"). You cannot set the parameters that the platform owns: seeds, threads, categorical declarations, and the loss or objective that a kind of uncertainty needs. For example, sklearn.tabular_quantiles sets the xgboost objective: reg:quantileerror. It rejects an objective that does not agree. The params models are a discriminated union. Thus the control plane validates them statically. For example, it rejects max_depth: -1 with a JSON pointer before a run starts.

Hyperparameters as run params v0.11¶

A fit logs the hyperparameters of the fitted estimator as run params with the prefix estimator., for example estimator.max_depth = 6. The fit logs only booleans, numbers and text (a maximum of 100 params). It does not log nested estimators, arrays or None. The Params tab of the run page shows them, and you can search runs by them (--filter 'param.estimator.max_depth >= 6'). See Track metrics, params and tags.

Uncertainty¶

Quantiles. xgboost fits all levels at the same time. The sklearn estimators that can do quantiles fit one model for each level. The platform sorts quantiles that cross (monotone rearrangement, which never increases the pinball loss). It reports the fraction of rows that cross as QUANTILE_CROSSING. If you do not set output.summary, the default (median, q5, q95) changes to agree with your grid. For example, with [0.1, 0.5, 0.9] you get median, q10, q90. If you request a level outside the grid, you get an error. The platform interpolates quantiles. It never extrapolates them.

Bootstrap. n replicates fit the estimator again on resampled rows. With by: store, they fit it on resampled whole stores. This is a cluster bootstrap that keeps the dependence in each store. expected outputs are the predictions of the replicates. predictive outputs add an out-of-bag residual for each cell. The draws are joint. Thus totals over stores, counterfactual deltas and all summaries are valid.

Split conformal. The platform keeps a calibration fraction out of the fit. By default, this fraction is the most recent weeks (calibration_by: last), as in a forecast. random selects rows at random. The radius is the ceil((n+1)(1-alpha))-th smallest absolute calibration residual. The calibration set can be too small for the requested alpha. Then you get a SETTINGS_INVALID error, not an infinite interval. refit: true fits the point model again on all rows after the calibration. The interval is then narrower, but the finite-sample guarantee is only approximate (not exact).

New stores, skus and markets¶

Score data can contain a label of a row dim that the fit did not see. Then the effective policy of the dim decides. This policy is the stricter of the policy of the data and the new_levels of the plugin:

pooled: the prediction is an average of the predictions that the row gets under each fitted level. The training frequency of each level is the weight. For a linear model with one-hot features, this is equal to a prediction with the average encoding. The platform calculates the average of quantiles and interval bounds (Vincentisation). It calculates the average of bootstrap draws for each draw, so that they stay joint.
zero: the level adds nothing. All one-hot columns are 0. The category codes are missing (for estimators that accept missing values natively).
error (or a fixed dim): UNSEEN_LEVELS before the run starts.

Contributions¶

The point plugins and the bootstrap plugins can return contributions(component, ...) next to y_hat:

exact (linear estimators. This is the default with contributions: auto): intercept plus coef × x for each column. The platform adds the one-hot columns of a coordinate feature into one component. The sum of the components is exactly equal to the prediction.
zeroing (all estimators): counterfactual zeroing. baseline is the prediction with all features at zero. The contribution of each feature is the decrease when only that feature is zero. interaction contains the remainder, so that the sum of the components stays correct. These contributions have the label counterfactual_zeroing. They are not Shapley values. SHAP is not in the scope of v1.

Contributions are additive in model space (output: {units: model}). Through a non-linear inverse, for example log1p, the inverse of each component is not a decomposition. If you want a sum of the parts that is correct, request model units.

Default plots and metrics v0.10¶

Each sklearn plugin now declares its standard plots. The fit run draws only the plots that the spec lists in outputs.plots. Thus, add the plots that you want to outputs.plots. In the spec editor, the Insert panel shows the plots of the model first.

outputs:
  plots: [core.calibration, core.roc_pr, core.confusion, ml.feature_importance]

An evaluate run from the Score page does not need this list. It draws the plots for the shape of each output: a classifier gets core.calibration, core.roc_pr and core.confusion.

Plugins	Plots
Regressor, quantiles, bootstrap, conformal	`core.fit_vs_actual`, `core.residuals_over_time`, `ml.feature_importance`
Classifier	`core.calibration`, `core.roc_pr`, `core.confusion`, `ml.feature_importance`

The three classifier plots read the probability in y_hat and the 0/1 target. They do not need draws. core.calibration shows a reliability diagram: do the predicted probabilities agree with how often the positive class occurs? core.roc_pr shows how well the model puts the positive rows before the negative rows. core.confusion counts the errors at the threshold 0.5. Refer to the plot guide.

Classifier metrics. The fit pairs the probability in y_hat with the target. Thus evaluate.metrics can include auc, brier and accuracy:

evaluate: {metrics: [auc, brier, accuracy]}

Before v0.10, the fit calculated no metric for a classifier. metrics.json was empty. The point metrics (rmse, mae and the other point metrics) use the probability too. Thus rmse is the root of the Brier score. Refer to Metrics in evaluate for the definitions.

A known limit

Validation refuses log_score for the classifier, because the plugin gives point predictions. Use brier. A regression target that has only the values 0 and 1 also gets the classifier plots on the Score page.

Multiclass classification v0.14¶

sklearn.multiclass_classifier puts each row into one of 3 or more classes. The target can be text ("cat") or whole numbers (3).

model:
  use: sklearn.multiclass_classifier@1
  settings:
    target: topic                       # text or whole-number labels
    rows: [ticket]
    time_kind: none                     # the rows have no time order
    columns: {roles: [covariate]}
    estimator: {kind: lightgbm.LGBMClassifier, params: {n_estimators: 300, num_leaves: 31}}
    max_classes: 50
outputs:
  plots: [core.roc_pr, core.confusion, ml.feature_importance]
evaluate: {metrics: [accuracy, brier, auc]}

Setting	Default	Meaning
`estimator`		A classification estimator of the table above
`class_order`	none	The classes in a fixed sequence. It must contain all the classes of the training data. Without it, the classes are in sorted sequence. A maximum of 200.
`max_classes`	50	The fit stops when the target has more classes (2 to 200)
`target_dtype`	`str`	The dtype of the target in the schema template: `str` or `int64`

The fit changes the labels to text. Whole numbers become 3, not 3.0. Rows with no label are not used. The output y_hat has a class dim with the class labels as coordinates. Each row of y_hat has the probabilities of the classes, and they add up to 1. y_hat has the attribute meaning: class_probability.

Plots and metrics. core.roc_pr draws one ROC curve and one precision-recall curve for each class against the other classes, and gives the macro AUC. core.confusion shows a K × K matrix of the most probable class. The metrics accuracy, brier and auc (the mean one-vs-rest AUC of the classes) use the class probabilities. As for the binary classifier, validation refuses log_score, because the plugin gives point predictions.

sklearn.tabular_classifier (binary) also accepts the new estimators.

Classify images v0.14¶

An image dataset (pixels(image, y, x, channel) and label(image)) can go to sklearn.multiclass_classifier after a step that changes each image into features:

From v0.15, the step dl.embed of mapi-jax gives the features of a frozen pretrained network (embedding(image, feature)). A linear head, for example sklearn.LogisticRegression, then learns the classes. This works with a few hundred labelled images. See Image features and a simple head.
Use a stratified split to hold out the same share of each class, or a column split to hold out the images that a manifest column marks. See Hold out images for a test.

For thousands of images, or to train all the weights of a network, use the models of Deep learning.

Persistence and safety¶

The bundle keeps the fitted estimators in its /native group. It never uses pickle:

scikit-learn estimators with skops. Before the platform loads the file, it lists the types in the file. It refuses all types that are not on the allow-list of that estimator. For example, a random forest can contain sklearn.tree._tree.Tree. A ridge regression can contain no extra types. The loaded object must be exactly the expected class.
xgboost models as the JSON document of the booster.
LightGBM models as the text model of the booster (v0.14). The text must start with tree. The platform loads it into a LightGBM booster, not into a pickled object.
CatBoost models as the JSON model (v0.14). The JSON must be an object with oblivious_trees.

The bundle manifest records the size, CRC32C, MD5 and SHA-256 of each file. The contract kit does a check that the bundle contains no entry that looks like pickle.

Write your own sklearn plugin¶

Implement estimator(). Then select a base class for the representation that you want:

from typing import ClassVar

from mapi_sklearn import PointSettings, SklearnModel, capabilities_for
from mapi_spec import Capabilities
from sklearn.linear_model import Ridge


class LinearDemand(SklearnModel[PointSettings]):
    id = "t.acme.linear_demand"
    version = "1.0.0"
    capabilities: ClassVar[Capabilities] = capabilities_for("none").model_copy(
        update={"contribution_scale": "additive"}  # a linear model decomposes exactly
    )

    def estimator(self, s: PointSettings) -> Ridge:
        return Ridge(alpha=0.5)  # must be exactly an allow-listed class

estimator() can return an instance from the allow-list, a {kind, params} mapping or a choice model. The platform rejects subclasses of estimators and parameters that are not in the params model. The base classes are SklearnModel (point), SklearnQuantileModel, SklearnBootstrapModel, SklearnConformalModel and SklearnClassifier.

Before you publish, run the contract kit on synthetic data:

from mapi_sdk.testing import ModelFixture, check_model
from mapi_sklearn.testing import demand_dataset, demand_registry, demand_schema, demand_spec

data = demand_dataset()  # a store x sku x week demand panel
report = check_model(
    LinearDemand,
    ModelFixture(
        spec=demand_spec("t.acme.linear_demand@^1", {"estimator": None}),
        data=data,
        raw_schema=demand_schema(data),
        registry=demand_registry(),
    ),
)
print(report.summary())
# ... [PASS] additive_contributions: contributions sum to 'y_hat' ...

Limits in v1¶

Forecasts are direct only. You must give the future features, or the pipeline must calculate them.
sklearn.tabular_classifier is binary. The positive class is the larger target value. For three or more classes, use sklearn.multiclass_classifier.
The multiclass classifier has no calibration plot.
The platform declares all row dims other than time_dim as categorical. Thus groups with integer labels (numeric store ids) need string labels.
Contributions are not available for the quantile plugins or the conformal plugins.