Skip to content

mapi_runner

This package is the run harness in the sandbox (see Runner). The supervisor modules import only mapi_spec and httpx. The worker modules run the SDK harness and the plugin code.

Job documents and input/output names

What RunSpec.spec holds for each run kind, and the input/output naming conventions.

The contract lives in :mod:mapi_spec.jobs (shared with the control plane, which writes the RunSpecs and reads the outputs); this module re-exports it for the runner's own modules.

Supervisor <-> worker channel

The supervisor <-> worker message channel (a socketpair; plan §8.7).

Frames are a 4-byte big-endian length followed by one UTF-8 JSON object. Every message is validated against the models below; the worker is untrusted, so anything malformed, oversized or of an unknown type is a SANDBOX_VIOLATION on the supervisor side.

Supervisor -> worker: exactly one :class:StartMessage (no HMAC key, no output URLs). Worker -> supervisor: state, progress, diagnostic, warning, metric, checkpoint, the tracking messages metrics (batched metric points), params, tags and user_log (structured log lines, stream user), media (v0.14: one file the worker wrote under {scratch}/media/), then exactly one terminal result or error. The supervisor accounts every tracking message against StartMessage.tracking (the run's :class:~mapi_spec.tracking.TrackingLimits).

StartMessage

Bases: StrictModel

Everything the worker needs, and nothing secret (no HMAC key, no write URLs).

StateMessage

Bases: StrictModel

ProgressMessage

Bases: StrictModel

DiagnosticMessage

Bases: StrictModel

WarningMessage

Bases: StrictModel

MetricMessage

Bases: StrictModel

CheckpointMessage

Bases: StrictModel

Checkpoint files were (re)written under {scratch}/checkpoints/.

ResultMessage

Bases: StrictModel

The run finished; every output is a regular file under {scratch}/outputs/.

ErrorMessage

Bases: StrictModel

The run failed (or was cancelled). Text from the worker is untrusted.

FrameReader 

FrameReader(max_bytes: int = MAX_WORKER_FRAME)

Non-blocking incremental frame decoder (the supervisor's side of the channel).

pending 

pending: bool

Bytes of an incomplete frame are buffered.

feed 

feed(data: bytes) -> list[bytes]

Add received bytes; return every complete frame body.

ProtocolError

Bases: Exception

A malformed, oversized or out-of-order frame.

Supervisor

The trusted supervisor (plan §8.1, §8.5, §8.7).

One supervisor runs per execution. It:

  1. reads MAPI_RUN_ID/MAPI_NONCE/MAPI_BOOTSTRAP_URL, fetches the RunSpec bootstrap object and deletes it before anything else happens;
  2. starts the untrusted worker (python -I -u -m mapi_runner.worker) as a child with a scrubbed environment, a socketpair for messages and pipes for stdout/stderr - as uid 10001 with no-new-privs when it runs as root, in dev mode otherwise;
  3. validates every worker message, turns them and the captured output into HMAC-signed events on its own stdout (the only thing ever written there), enforces the log, event, tracking, scratch and output caps, and coalesces progress;
  4. handles SIGTERM and the run deadline (forward to the worker, wait for the checkpoint flush, upload a partial manifest);
  5. uploads the media the worker logs (v0.14: each file under {scratch}/media/ is read safely, hashed and checked by the supervisor, then sent through the run's media POST policy) and emits one media event per item;
  6. uploads outputs with the signed POST policy (streaming, hashing as it goes), writes the signed :class:~mapi_spec.run.OutputManifest with the signed PUT and exits with a :class:~mapi_spec.run.RunnerExit code.

The supervisor never imports plugin code, the SDK, numpy or xarray.

EXIT_FOR_CLASS 

EXIT_FOR_CLASS: Final[Mapping[FailureClass, RunnerExit]] = {
    FailureClass.USER: RunnerExit.USER_ERROR,
    FailureClass.RESOURCE_EXHAUSTED: RunnerExit.RESOURCE_EXHAUSTED,
    FailureClass.INFRA: RunnerExit.INFRA_ERROR,
    FailureClass.INTEGRITY: RunnerExit.INTEGRITY_ERROR,
    FailureClass.TIMEOUT: RunnerExit.TIMEOUT,
    FailureClass.SANDBOX: RunnerExit.SANDBOX_VIOLATION,
}

Supervisor 

Supervisor(
    config: SupervisorConfig,
    environ: Mapping[str, str],
    *,
    event_fd: int,
    client: Client | None = None,
    isolation: IsolationPlan | None = None,
)

Runs one execution; :meth:run returns the process exit code.

SupervisorConfig 

SupervisorConfig(
    worker_python: str = executable,
    worker_registry: str | None = None,
    scratch_root: str | None = None,
    keep_scratch: bool = False,
    term_grace_s: float = 5.0,
    deadline_reserve_s: float = 15.0,
    scratch_check_s: float = 2.0,
    base_image: str | None = None,
    overlay_hash: str | None = None,
)

How the worker is started and how long things may take.

worker_registry 

worker_registry: str | None = None

module:callable returning the worker's plugin registry (default: entry points).

term_grace_s 

term_grace_s: float = 5.0

Time the worker gets to stop after SIGTERM before it is killed.

deadline_reserve_s 

deadline_reserve_s: float = 15.0

Time kept before the deadline for the checkpoint flush and the partial manifest.

Worker isolation

Worker process isolation inside the execution (plan §8.1, §9).

When the supervisor runs as root (Cloud Run gen2 / docker), the worker is started as uid/gid 10001 with no supplementary groups, PR_SET_NO_NEW_PRIVS and no core dumps, and the supervisor makes itself non-dumpable so the worker cannot ptrace it. Anywhere else (macOS, a developer's non-root shell) the uid split is impossible: the run proceeds in dev mode and the supervisor says so in a warning event. Dev mode is never a security boundary.

IsolationPlan 

IsolationPlan(
    drop_privileges: bool,
    no_new_privs: bool,
    reason: str | None = None,
)

How the worker will be started.

reason 

reason: str | None = None

Why isolation is reduced (dev mode); None when fully isolated.

preexec 

preexec() -> Callable[[], None]

Runs in the forked worker before exec: no core dumps, no-new-privs.

plan_isolation 

plan_isolation(
    *, euid: int | None = None, platform: str | None = None
) -> IsolationPlan

Decide how to isolate the worker on this host.

harden_supervisor 

harden_supervisor() -> None

Mark the supervisor non-dumpable (no ptrace/core from same-uid processes).