finprint.fingerprint

module finprint

Canonical-rows fingerprint, file hashes and the write-and-read-back helpers.

The rule (basis canonical_rows/ump_recipe-2) is chosen so a frame and the same frame written to CSV and read back give the same fingerprint, for the dtypes CSV can carry (integers, floats, booleans, text, dates and datetimes):

1. A named index (any level with a name) becomes ordinary columns first (beside a column of the same name, if there is one); an unnamed index is ignored, because rec.save writes it only when it is named.
2. Column names become strings and are sorted by name (a stable sort keeps duplicates in their order).
3. Row order is kept: in a weekly panel the order is part of the data.
4. Values are normalised: NaN, None, NaT, pd.NA and the empty string become null; integral numbers below 2**53 become integers (so 3 and 3.0 agree); other floats are kept exactly (shortest round-trip form, so a change in the last digit, such as a millisecond in an epoch timestamp, changes the fingerprint; basis -1 rounded to 12 significant digits and hid such edits); booleans stay booleans; datetimes become ISO text with a space separator, the time dropped at midnight, trailing zeros of the fraction dropped; text that already looks like such a datetime is normalised the same way; anything else is its str.
5. Each column is hashed as json([name, values]) into one running sha256 over the sorted columns.

The result is a bare 64-character sha256 hex.

catalogue_fingerprint is a second fingerprint, recorded on every input as catalogue_fingerprint: exactly the data catalogue’s rule (ump_spec.rules.canonical_rows_fingerprint, basis canonical_rows), so apps/data can match an input to a connected dataset whose sha256 was computed from the same rows. Each cell becomes a plain JSON value the way apps/data’s connectors make them (connectors.base.plain_cell): NaN, None, NaT, pd.NA and infinities are null, numpy numbers become Python numbers, datetimes and dates become isoformat() text, timedeltas PT<seconds>S, Decimals floats, bytes hex, anything else its str; the empty string stays the empty string. Then columns are sorted by name (cells permuted to match), rows sorted by their canonical JSON, and {"columns": [...], "rows": [...]} hashed as canonical JSON (sorted keys, no whitespace, UTF-8, integral floats below 2**53 as integers). Column order and row order do not count; any changed value, row or column does. A named index becomes columns first, as above. tests/test_spec_alignment.py checks it equals ump_spec’s function whenever ump_spec is installed; this package never imports it.

Steps are compared with step_hash instead (basis memory_hash/ump_recipe-1): a vectorised hash of the in-memory values (pandas.util.hash_pandas_object), column and index names and shape. It costs a fraction of a second on a million-row panel where the canonical rule costs seconds, and it is only ever compared with another in-memory object (replay on the same machine), never with a file. Text that looks numeric, or the literal strings pandas reads as missing (“NA”, “null”), do not survive a CSV round trip and are outside the rule.

Functions

sha256_bytes

function

def sha256_bytes(data: bytes) -> str

Source: packages/finprint/src/finprint/fingerprint.py, line 63

sha256_file

function

def sha256_file(path: str | Path) -> str

Source: packages/finprint/src/finprint/fingerprint.py, line 67

normalise_value

function

def normalise_value(v: Any) -> Any

One cell under the canonical-rows rule.

Source: packages/finprint/src/finprint/fingerprint.py, line 98

as_frame

function

def as_frame(obj: Any) -> pd.DataFrame

A Series becomes a one-column frame (named value when unnamed).

Source: packages/finprint/src/finprint/fingerprint.py, line 143

index_is_named

function

def index_is_named(df: pd.DataFrame) -> bool

Source: packages/finprint/src/finprint/fingerprint.py, line 152

fingerprint

function

def fingerprint(obj: pd.DataFrame | pd.Series) -> str

Canonical-rows fingerprint (module docstring has the rule).

Source: packages/finprint/src/finprint/fingerprint.py, line 156

plain_cell

function

def plain_cell(v: Any) -> Any

One cell as the plain JSON value the catalogue’s rule hashes (module docstring), already canonical.

Source: packages/finprint/src/finprint/fingerprint.py, line 175

catalogue_fingerprint

function

def catalogue_fingerprint(obj: pd.DataFrame | pd.Series) -> str

The data catalogue’s canonical-rows fingerprint of obj (module docstring has the rule).

Source: packages/finprint/src/finprint/fingerprint.py, line 229

hashes

function

def hashes(obj: pd.DataFrame | pd.Series) -> tuple[str, str]

(step_hash, state_hash) of obj from one pass over its values.

Source: packages/finprint/src/finprint/fingerprint.py, line 280

step_hash

function

def step_hash(obj: pd.DataFrame | pd.Series) -> str

In-memory hash a step’s output is recorded and replayed with (module docstring).

Source: packages/finprint/src/finprint/fingerprint.py, line 290

state_hash

function

def state_hash(df: pd.DataFrame) -> str

Fast change detector for in-place edits, including dtypes and index or column level names (never stored).

Source: packages/finprint/src/finprint/fingerprint.py, line 295

columns_of

function

def columns_of(obj: pd.DataFrame | pd.Series) -> dict[str, str]

Source: packages/finprint/src/finprint/fingerprint.py, line 300

output_format

function

def output_format(path: str | Path) -> str

Source: packages/finprint/src/finprint/fingerprint.py, line 305

frame_bytes

function

def frame_bytes(df: pd.DataFrame, fmt: str = 'csv') -> tuple[bytes, bool]

Bytes rec.save writes, and whether the index was written (only when it is named).

Source: packages/finprint/src/finprint/fingerprint.py, line 309

read_back

function

def read_back(data: bytes, fmt: str = 'csv') -> pd.DataFrame

Source: packages/finprint/src/finprint/fingerprint.py, line 319

round_trip_fingerprint

function

def round_trip_fingerprint(
    df: pd.DataFrame | pd.Series,
    fmt: str = 'csv',
) -> tuple[str, bytes, bool]

Fingerprint of the frame as it would be written and read back, plus the bytes and the index flag.

Source: packages/finprint/src/finprint/fingerprint.py, line 328