ump_spec.secrets

module ump-spec

Secret hygiene for data connections (POC blueprint DATA-CATALOGUE §3, spec §18 and §20).

Secrets are stored by reference only (secret://<client_id>/<name>). They never appear in artifact docs, events, logs, lineage, Merkle leaves or API responses. find_secret_fields is the guard the contract models run on every connection doc and every connected dataset’s request spec; apps run it (or redact_secrets) before they log, emit or return anything built from connector config.

A field counts as a secret when its key names a credential (password, token, api_key, authorization, …) and its value is a non-empty string that is not a secret:// reference, or when any string value is a URL carrying a password (postgresql://user:pw@host/db) or a credential in its query string (?token=...). Booleans, numbers and secret:// references are never flagged.

find_secret_text is the shape scan for free text that is not a key/value config: notebook cell source, a named step’s code and its argument strings in a transform recipe (TRANSFORM-RECIPES §4 and §6). It flags Bearer/Basic tokens, an Authorization header with a literal value, known key prefixes (sk_live_, AKIA, ghp_, JSON web tokens …), secret-looking assignments (api_key = '...'), passwords in SQL or key=value strings, credential query parameters and email addresses (personal data, never stored in a recipe). <redacted> and [redacted] markers are safe. The patterns are the server’s copy of finprint.redact (the analyst package cannot depend on ump_spec); packages/finprint/tests/ test_spec_alignment.py checks that every sample the client warns about is refused here too. The client’s looser heuristics (any mixed-case quoted literal, long hex literals) stay client-side warnings because column names and fingerprints look like them.

Functions

find_secret_fields

function

def find_secret_fields(
    obj: Any,
    ptr: str = '',
    *,
    name_maps: frozenset[str] | set[str] = frozenset(),
) -> list[str]

JSON pointers of every value that looks like a secret held in clear. Empty means clean.

name_maps names keys whose object value maps names to values (a recipe’s columns: column name to dtype). Inside such a map the key-name rule is skipped, so a column called cookie or pass is not a secret; the shape rules still run on its keys and values.

Source: packages/ump-spec/src/ump_spec/secrets.py, line 93

find_secret_text

function

def find_secret_text(text: str) -> list[str]

What kinds of secret or personal data text holds in clear, by shape. Empty means clean.

Source: packages/ump-spec/src/ump_spec/secrets.py, line 71

is_secret_ref

function

def is_secret_ref(value: Any) -> bool

A secret:// reference, or the [redacted] marker: both are safe to show.

Source: packages/ump-spec/src/ump_spec/secrets.py, line 80

redact_secrets

function

def redact_secrets(obj: Any) -> Any

A copy of obj with every value find_secret_fields would flag replaced by [redacted]. Use it before logging or returning connector config or a request spec.

Source: packages/ump-spec/src/ump_spec/secrets.py, line 132

Constants

Name	Type	Value	Description
EMAIL_RE		re.compile('[A-Za-z0-9._%+-]+@[A-Za-z0-9-]+(?:\\.[A-Za-z0-9-]+)*\\.[A-Za-z]{2...
KEY_PREFIX_RE		re.compile('(?:sk|pk|rk)[-_](?:live|test|proj)?[-_]?[A-Za-z0-9]{16,}|AKIA[0-9...
REDACTED		'[redacted]'
SECRET_KEYS	frozenset[str]	frozenset({'password', 'passwd', 'pwd', 'pass', 'secret', 'client_secret', 's...	Normalised key names (lower case, non-alphanumerics as _) whose string values are credentials.
SECRET_REF_RE		'^secret://[A-Za-z0-9_.\\-]+/[A-Za-z0-9_.\\-/]+$'	secret://<client_id>/<name>; production maps it to a GCP Secret Manager name.