finprint
package finprint
finprint: record how a dataset was made in pandas, save the recipe beside the output, replay it to check, apply it to new data of the same shape.
%load_ext finprint # notebooks: also records cell historyimport finprint as fprec = fp.session("northwind weekly prep")raw = rec.read_csv("northwind_raw.csv")clean = raw.drop(index=[12, 13]) # recordedrec.save(clean, "northwind_weekly_clean.csv") # writes the .csv and northwind_weekly_clean.recipe.jsonBlueprint: docs/poc/TRANSFORM-RECIPES.md. Works without IPython.
Read the finprint guide first for the main entry points.
Modules
Section titled “Modules”| Module | What it holds |
|---|---|
finprint.apply | Apply: run a saved recipe on new inputs of the same shape (more weeks, an extra column), not to check it. |
finprint.catalogue | Thin HTTP client for apps/data (port 8108): read a catalogued dataset, publish an output with its recipe. |
finprint.cli | finprint: show, verify, apply, export and schema. |
finprint.codes | Note codes a recipe can carry (blueprint docs/poc/TRANSFORM-RECIPES.md sections 3 to 6). |
finprint.describe | Plain-language summary of a recipe (finprint show): “Dropped 2 rows: 156 to 154”, “Added spend_net”. |
finprint.encoding | Tagged JSON for step arguments and series expressions, and the inverse used by replay. |
finprint.export | Export a recipe in existing wire formats (load-bearing decision 9): an OpenLineage RunEvent with schema and columnLineage facets, and W3C PROV-JSON with entities, activities, used and wasGeneratedBy. |
finprint.fingerprint | Canonical-rows fingerprint, file hashes and the write-and-read-back helpers. |
finprint.notebook | IPython extension: %load_ext finprint. |
finprint.recording | The recording session (rec): readers, named steps, save, publish. |
finprint.redact | Privacy rules for what goes into a recipe (blueprint section 4). |
finprint.replay | Replay check: rerun a recipe from its inputs and compare fingerprints step by step. |
finprint.schema | The recipe file, <output>.recipe.json (schema ump.transform_recipe/0.1). |
finprint.stepcode | What a named step’s code needs besides its arguments, checked when the step runs. |
finprint.tracked | Proxies that record what is done to a frame: TrackedFrame, TrackedSeries and their helpers. |
finprint.trust | Which named-step code this machine recorded, so replay runs only that code without asking. |
Re-exports
Section titled “Re-exports”Documented on the page of the module that defines them.
| Name | Kind | Summary |
|---|---|---|
APPLY_CODES | constant | |
CODES | constant | |
FINGERPRINT_BASIS | constant | |
NOTE_CODES | constant | |
SCHEMA_VERSION | constant | |
ApplyCode | class | |
ApplyResult | class | |
CatalogueError | class | |
NoteCode | class | |
RecipeError | class | |
RecipePrivacyWarning | class | Cell source or step code that looks like it holds an email address or a key. |
RecipeWarning | class | A recipe was saved with a note the analyst should see (partial capture, transient data). |
ReplayResult | class | |
SaveResult | class | |
Session | class | Records reads, verbs and named steps on tracked frames. Create one with finprint.session(title). |
TrackedBase | class | Base of the proxies, so the encoder can recognise them without importing tracked. |
TrackedDeferred | class | A tracked groupby (or rolling, resample, expanding, ewm): the next aggregation is recorded as one step. |
TrackedFrame | class | A DataFrame proxy bound to a session. Every frame version has an id (in1, f7…). |
TrackedSeries | class | A Series proxy that remembers its expression over frame versions (never recorded as a step by itself). |
TransformRecipe | class | <output>.recipe.json: inputs by hash, steps, notebook cells, the output’s fingerprint and the analyst’s own replay check. |
TransientDataError | class | |
apply_verification | function | |
column_origins | function | For each output column, the input columns it came from, and a note for columns of unknown origin. |
current | function | |
json_schema | function | JSON Schema 2020-12 for TransformRecipe (the form the published file holds). |
json_schema_text | function | |
load_recipe | function | |
round_trip_fingerprint | function | Fingerprint of the frame as it would be written and read back, plus the bytes and the index flag. |
row_bound_steps | function | Steps whose recorded arguments address rows by literal position or label, so they cannot carry over to new data ([{"step_id", "op", "reason"}]). Pure: reads only the recorded op and arguments. |
data_derived_steps | function | Steps that use a literal the analyst took out of a value computed from the data (float(s.median()), int(s.max()), f"{s.mean():.1f}", np.float64(x), math.log(x)) or that follow an if on such a value. The recorder marks them (args.data_derived: how the value was taken out); on new data they would reuse the recorded value, so apply refuses them. Values kept as they come from pandas are recorded as expressions ({"__scalar__": ...}) and recomputed instead. |
session | function | Start a recording session and make it current. In a notebook where %load_ext finprint started an empty session, that one is renamed and returned so the cells recorded so far stay with it. |
sha256_file | function | |
step | function | @finprint.step: like @rec.step but bound to whichever session is current when the function runs. |
summary | function | Multi-line text for finprint show. |
to_openlineage | function | An OpenLineage 2-0-2 RunEvent (eventType COMPLETE) for the recipe. |
to_prov | function | W3C PROV-JSON: inputs, intermediate frames and the output as entities, steps as activities. |
untrack | function | The plain pandas object behind a tracked one (the same object, not a copy). Edits made to it are not recorded; the session notices them as unexplained changes and marks the recipe partial. |
verify | function | Replay the recipe at path and, unless dry_run, write the verification back into the file. |
write_recipe | function |
Functions
Section titled “Functions”load_ipython_extension
Section titled “load_ipython_extension”function
def load_ipython_extension(ip: Any) -> None%load_ext finprint.
Source: packages/finprint/src/finprint/__init__.py, line 54
unload_ipython_extension
Section titled “unload_ipython_extension”function
def unload_ipython_extension(ip: Any) -> NoneSource: packages/finprint/src/finprint/__init__.py, line 59
Constants
Section titled “Constants”| Name | Type | Value | Description |
|---|---|---|---|
__version__ | '0.2.0' |