Container images v0.2¶

The platform has seven images (and a worker target). The build makes them from images/<name>/Dockerfile, with the repository root as the build context. The images have three purposes:

To run the exact production artifacts locally (Docker Compose).
To support the docker executor.
To become the Cloud Run Jobs in v0.3.

make docker-build                      # every image, tagged mapi/<name>:<version>
make docker-build IMAGES="server web"  # a subset
make docker-pin-check                  # every base image pinned by digest?

The images¶

Image	Contents	Runs as	Entry point
`mapi/base-bayes`	`mapi-runner`, SDK, steps, plot recipes, workflow nodes, `mapi-pymc` with the `pathfinder` extra, the `mmm.`, `tabular.` and `sci.*` plugins (`mapi-plugin-science`, which uses SciPy), `mapi-jax` for the CPU (v0.15), a C++ compiler for PyTensor	`10001`	`mapi-runner`
`mapi/base-ml`	`mapi-runner`, SDK, steps, plot recipes, workflow nodes, `mapi-sklearn` with the `gbm` extra (`sklearn.` plugins, LightGBM and CatBoost from v0.14) and `libgomp1` (for LightGBM), `mapi-jax` for the CPU (v0.15). No PyMC.*	`10001`	`mapi-runner`
`mapi/base-dl` (v0.15)	`mapi-runner`, SDK, steps, plot recipes, workflow nodes, `mapi-jax` with the `cuda12` extra (the `dl.` models and the step `dl.embed` on JAX, Flax and Optax, with the NVIDIA CUDA 12 wheels). No PyMC, no scikit-learn, no C++ compiler.*	`10001`	`mapi-runner`
`mapi/base-forecast`	`mapi-runner`, SDK, steps, plot recipes, workflow nodes, `mapi-pymc` with the `pathfinder` extra, the `ts.` plugins (`mapi-plugin-ts` and `mapi-plugin-statsmodels`), the `causal.` plugins (`mapi-plugin-causal`) and pymc-extras	`10001`	`mapi-runner`
`mapi/worker`	`base-bayes` with the `ts.` and `causal.` plugins and `mapi-server` (target `worker` of the base-bayes Dockerfile). Only for compose	`10001`	`mapi-server worker`
`mapi/server`	The control plane: `mapi-server`, `mapi-spec`, the JSON-only `mapi-plots` core, the JSON-only `mapi-workflows` engine, docker-py. No science dependencies.	`10001`	`mapi-server api` (or `worker`, `migrate`)
`mapi/gcs-shim`	The local Cloud Storage emulator	`10001`	`mapi-gcs-shim`
`mapi/web`	nginx that serves `apps/web` (port 8080) and `apps/plot-sandbox` (port 8081)	`101`	nginx

causal.impact uses the ts.bsts model of mapi-plugin-ts. Thus the causal.* plugins are in mapi/base-forecast, and mapi/base-bayes does not contain them. pymc-extras (state space models and the Pathfinder engine) adds preliz, Numba and Matplotlib. Thus mapi/base-forecast is larger than in v0.12. mapi/base-bayes also contains pymc-extras (the Pathfinder engine).

From v0.14, mapi-runner depends on Pillow (for the image_folder ingest). Thus all runner images contain Pillow. Pillow decodes uploaded images only in the sandboxed ingest run, never in the control plane. LightGBM and CatBoost (about 150 MB) are only in mapi/base-ml. tabular.image_logistic runs in mapi/base-bayes and does not need Pillow.

The deep learning image v0.15¶

mapi/base-dl runs the dl.* models of mapi-jax. JAX comes with its CUDA 12 wheels (jax-cuda12-plugin and the NVIDIA runtime wheels from pip) on the same python:3.12-slim base. There is no NVIDIA base image. The CUDA wheels make the image about 3 GB. Thus it is the largest image.

Without a GPU, JAX uses the CPU. Each dl.* model continues to train, but slowly. The runner sends the warning GPU_UNAVAILABLE when a run of the class GPU sees no NVIDIA device (Troubleshooting).
The image sets XLA_PYTHON_CLIENT_PREALLOCATE=false (JAX gets GPU memory when it needs it) and XLA_FLAGS=--xla_gpu_deterministic_ops=true (a fit with a seed gives the same result again). The build arguments MAPI_XLA_PREALLOCATE and MAPI_XLA_FLAGS set them. The other images leave them empty.
mapi/base-bayes and mapi/base-ml contain mapi-jax for the CPU. Thus the step dl.embed can run in front of their models, for example tabular.image_logistic or sklearn.multiclass_classifier.

Not verified

Nobody has built or run mapi/base-dl on a GPU. Do a test with a small run of the class GPU before you use it. Make sure that the param dl.backend of the run is gpu. Refer to Docker Compose and GPU jobs.

The guide Deep learning tells how to use the models.

make docker-build IMAGES="server web" builds a subset. make docker-build builds all images. It also builds mapi/worker when base-bayes is in the set. Each image has the OCI labels title, version, revision (git commit) and created.

How the Python images are built¶

Each Python image uses the same multi-stage pattern:

manifests copies the build context. It deletes all files except the pyproject.toml and uv.lock files. Thus its output changes only when dependencies change.
builder installs the third-party dependencies of the package set of the image. It uses only those manifests (uv sync --frozen --no-install-workspace, one time for each package, into /opt/mapi/venv). BuildKit keeps this layer in a cache. Thus, when you edit source code, the build does not install PyMC again. The builder then copies the sources and installs the workspace packages, non-editable. Each version comes from uv.lock.
runtime starts again from the same python:3.12-slim base. It adds tini (and g++ for the PyMC images) and makes user 10001. Then it copies in only the virtual environment.

The images that run mapi-server (mapi/server and mapi/worker) also copy uv.lock to /opt/mapi/uv.lock and set MAPI_UV_LOCK_PATH to it. Its SHA-256 is the image digest. The API and the orchestrator use this digest as the cache key of the step and model catalog. make dev calculates the same value. The build makes the two images from one lock, so api and worker agree.

The package set is the MAPI_PACKAGES build argument. images/common/install_packages.py changes it into uv sync calls. The build never installs the xgboost dependency nvidia-nccl-cu13 (MAPI_NO_INSTALL). This is a 300 MB wheel that only multi-GPU training uses.

Build-time guards¶

If an image is not what it claims to be, the build fails:

Package assertions. images/common/assert_dists.py does a check of MAPI_ASSERT_PRESENT and MAPI_ASSERT_ABSENT against the real environment. For example, base-ml must not contain pymc, pytensor or numba, and must contain pillow, lightgbm and catboost (v0.14). base-bayes and base-forecast must contain pillow and must not contain lightgbm or catboost. server must not contain numpy, xarray, pandas, scipy, pyarrow, PyMC, pillow, lightgbm, catboost, mapi_sdk, mapi_runner or a plugin. From v0.15, base-bayes and base-ml must contain mapi-jax, flax and optax, and base-forecast must not contain them. base-dl must contain jax, jaxlib, flax, optax and jax-cuda12-plugin, and must not contain mapi-pymc, mapi-sklearn, mapi-plugin-mmm, mapi-plugin-tabular, mapi-plugin-ts, mapi-plugin-causal, mapi-plugin-science, torch or tensorflow.
Import contracts in the control-plane image. The contracts stage of images/server/Dockerfile installs import-linter (hash-pinned from uv.lock) into a temporary copy of the environment. It runs the contracts of the repository against the packages exactly as they are installed. The runtime stage depends on that stage, so BuildKit cannot skip it.
Catalog export. The runner images load each installed step, model and plot plugin. They write /opt/mapi/catalog.json with its digest (/opt/mapi/catalog.digest). If no steps or models are registered, the build fails.

We also prove the same claims without Docker. tests/devops/test_container_invariants.py calculates the dependency closure of each image from uv.lock (scripts/lock_closure.py). Then it does a check of the closure against the assertions.

Compile caches and the read-only root filesystem¶

PyTensor compiles C at run time, and Numba keeps machine code in a cache. During the build, a small hierarchical model compiles and fills the two caches. The build then puts the caches read-only into /opt/mapi/cache. PyTensor needs a writable compile directory, because it takes a lock and adds new modules.

Thus, at start-up, the entry point (images/common/runner-entrypoint.sh) builds a symlink farm of the cache under /tmp/mapi-cache. The farm has real directories, with files that are symlinks to the objects in the cache. This uses some KiB of RAM. This is important on the in-memory filesystem of Cloud Run. PyTensor accepts the moved entries and does not compile them again.

HOME, PYTENSOR_FLAGS, NUMBA_CACHE_DIR and MPLCONFIGDIR all point under /tmp. The first three are on the worker-environment allow-list of the runner (mapi_spec.run.WORKER_ENV_ALLOWLIST). The allow-list also has XLA_FLAGS. From v0.15, it also has JAX_PLATFORMS, XLA_PYTHON_CLIENT_PREALLOCATE, XLA_PYTHON_CLIENT_MEM_FRACTION and CUDA_VISIBLE_DEVICES, for the device and the GPU memory of JAX. Thus plugin code sees them after the supervisor removes the other variables from the environment. As a result, each image runs with --read-only and a tmpfs on /tmp.

docker run mapi/base-bayes runs the runner. The image gives arguments that start with - to the runner. All other commands replace the runner (docker run mapi/base-bayes python -c ...).

The web image and the plot sandbox¶

mapi/web builds the two Vite apps with pnpm. It uses --frozen-lockfile and --ignore-scripts, so no package lifecycle scripts run. One unprivileged nginx with two server blocks serves the apps. NGINX_ENVSUBST_TEMPLATE_DIR selects which blocks nginx uses: /etc/nginx/templates/all (the default), web or plots. Origins are runtime settings. At start-up, nginx puts them into the headers.

	App (`:8080`, `MAPI_WEB_ORIGIN`)	Plot sandbox (`:8081`, `MAPI_PLOTS_ORIGIN`)
CSP	`default-src 'self'`; `script-src 'self'` (no `unsafe-eval`, no inline scripts); `frame-src` = the plots origin; `connect-src` = self, `MAPI_API_URL`, `MAPI_GCS_ENDPOINT`; `frame-ancestors 'none'`	`default-src 'none'`; scripts, styles and images only from its own origin; `connect-src` = only its own `/topojson/`; `frame-ancestors` = the app origin
Other headers	`X-Frame-Options: DENY`, `nosniff`, `Referrer-Policy: no-referrer`, COOP and CORP `same-origin`, Trusted Types in report-only mode	`nosniff`, `no-referrer`, `Access-Control-Allow-Origin: *`
API access	nginx sends `/v1/` to `MAPI_API_UPSTREAM`, unbuffered, with a read timeout of one hour for SSE	None: no proxy, and the CSP blocks it

The app frames the sandbox documents with sandbox="allow-scripts" and no same-origin access. Thus they have an opaque origin, and the browser gets their module scripts, styles and topojson in CORS mode. For this reason, the sandbox answers with Access-Control-Allow-Origin: *. The assets are static and public, and there are no cookies. For the same reason, its CSP lists the plots origin explicitly next to 'self'.

MAPI_WEB_SCRIPT_SRC_EXTRA (empty by default) adds sources to the script-src of the app. Use it only after a review, for example if a library needs 'wasm-unsafe-eval'. Trusted Types stay in report-only mode until the SPA has its policy.

Base images and pinning¶

Each base image is a build argument with a readable tag and a digest:

ARG PYTHON_IMAGE=python:3.12-slim-trixie@sha256:a6e3…11b1

The Dockerfile frontend (# syntax=) and the compose postgres image are pinned in the same way. make docker-pin resolves each tag again through the registry API. It needs no Docker daemon. Examine the diff that it makes as you examine each dependency update. make docker-pin-check fails if an image is not pinned. CI and make release-check also run it.

Licences in the images¶

Each image has the licence files of the project and THIRD-PARTY-NOTICES.md in /usr/share/doc/mapi/. The notices list each third-party Python and JavaScript package in the image set, with its version and licence. They also contain the NOTICE files, and the licence texts of the JavaScript packages that the build compiles into the web app. When uv.lock or pnpm-lock.yaml changes, run make notices to make the file again. While the file is not current, the test suite and make release-check fail. Refer to Licensing.

CI¶

.github/workflows/ci.yml builds each image, and the worker target, on each push and pull request. It uses the GitHub Actions layer cache. CI never pushes the images. A separate job does a check of the digest pins. It also validates the two compose files with docker compose config.

You can do a smoke test when you have built the images and a Docker daemon runs. Then uv run pytest tests/devops/test_compose_smoke.py -m docker starts postgres, gcs-shim, migrate, api and worker under a temporary project name. It publishes no host ports. Then it makes sure that api and worker started. It also makes sure that they agree on the values that bundle verification needs: the image digest and the bundle-signing key.