Skip to content

finprint

package finprint

finprint: record how a dataset was made in pandas, save the recipe beside the output, replay it to check, apply it to new data of the same shape.

%load_ext finprint # notebooks: also records cell history
import finprint as fp
rec = fp.session("northwind weekly prep")
raw = rec.read_csv("northwind_raw.csv")
clean = raw.drop(index=[12, 13]) # recorded
rec.save(clean, "northwind_weekly_clean.csv") # writes the .csv and northwind_weekly_clean.recipe.json

Blueprint: docs/poc/TRANSFORM-RECIPES.md. Works without IPython.

Read the finprint guide first for the main entry points.

ModuleWhat it holds
finprint.applyApply: run a saved recipe on new inputs of the same shape (more weeks, an extra column), not to check it.
finprint.catalogueThin HTTP client for apps/data (port 8108): read a catalogued dataset, publish an output with its recipe.
finprint.clifinprint: show, verify, apply, export and schema.
finprint.codesNote codes a recipe can carry (blueprint docs/poc/TRANSFORM-RECIPES.md sections 3 to 6).
finprint.describePlain-language summary of a recipe (finprint show): “Dropped 2 rows: 156 to 154”, “Added spend_net”.
finprint.encodingTagged JSON for step arguments and series expressions, and the inverse used by replay.
finprint.exportExport a recipe in existing wire formats (load-bearing decision 9): an OpenLineage RunEvent with schema and columnLineage facets, and W3C PROV-JSON with entities, activities, used and wasGeneratedBy.
finprint.fingerprintCanonical-rows fingerprint, file hashes and the write-and-read-back helpers.
finprint.notebookIPython extension: %load_ext finprint.
finprint.recordingThe recording session (rec): readers, named steps, save, publish.
finprint.redactPrivacy rules for what goes into a recipe (blueprint section 4).
finprint.replayReplay check: rerun a recipe from its inputs and compare fingerprints step by step.
finprint.schemaThe recipe file, <output>.recipe.json (schema ump.transform_recipe/0.1).
finprint.stepcodeWhat a named step’s code needs besides its arguments, checked when the step runs.
finprint.trackedProxies that record what is done to a frame: TrackedFrame, TrackedSeries and their helpers.
finprint.trustWhich named-step code this machine recorded, so replay runs only that code without asking.

Documented on the page of the module that defines them.

NameKindSummary
APPLY_CODESconstant
CODESconstant
FINGERPRINT_BASISconstant
NOTE_CODESconstant
SCHEMA_VERSIONconstant
ApplyCodeclass
ApplyResultclass
CatalogueErrorclass
NoteCodeclass
RecipeErrorclass
RecipePrivacyWarningclassCell source or step code that looks like it holds an email address or a key.
RecipeWarningclassA recipe was saved with a note the analyst should see (partial capture, transient data).
ReplayResultclass
SaveResultclass
SessionclassRecords reads, verbs and named steps on tracked frames. Create one with finprint.session(title).
TrackedBaseclassBase of the proxies, so the encoder can recognise them without importing tracked.
TrackedDeferredclassA tracked groupby (or rolling, resample, expanding, ewm): the next aggregation is recorded as one step.
TrackedFrameclassA DataFrame proxy bound to a session. Every frame version has an id (in1, f7…).
TrackedSeriesclassA Series proxy that remembers its expression over frame versions (never recorded as a step by itself).
TransformRecipeclass<output>.recipe.json: inputs by hash, steps, notebook cells, the output’s fingerprint and the analyst’s own replay check.
TransientDataErrorclass
apply_verificationfunction
column_originsfunctionFor each output column, the input columns it came from, and a note for columns of unknown origin.
currentfunction
json_schemafunctionJSON Schema 2020-12 for TransformRecipe (the form the published file holds).
json_schema_textfunction
load_recipefunction
round_trip_fingerprintfunctionFingerprint of the frame as it would be written and read back, plus the bytes and the index flag.
row_bound_stepsfunctionSteps whose recorded arguments address rows by literal position or label, so they cannot carry over to new data ([{"step_id", "op", "reason"}]). Pure: reads only the recorded op and arguments.
data_derived_stepsfunctionSteps that use a literal the analyst took out of a value computed from the data (float(s.median()), int(s.max()), f"{s.mean():.1f}", np.float64(x), math.log(x)) or that follow an if on such a value. The recorder marks them (args.data_derived: how the value was taken out); on new data they would reuse the recorded value, so apply refuses them. Values kept as they come from pandas are recorded as expressions ({"__scalar__": ...}) and recomputed instead.
sessionfunctionStart a recording session and make it current. In a notebook where %load_ext finprint started an empty session, that one is renamed and returned so the cells recorded so far stay with it.
sha256_filefunction
stepfunction@finprint.step: like @rec.step but bound to whichever session is current when the function runs.
summaryfunctionMulti-line text for finprint show.
to_openlineagefunctionAn OpenLineage 2-0-2 RunEvent (eventType COMPLETE) for the recipe.
to_provfunctionW3C PROV-JSON: inputs, intermediate frames and the output as entities, steps as activities.
untrackfunctionThe plain pandas object behind a tracked one (the same object, not a copy). Edits made to it are not recorded; the session notices them as unexplained changes and marks the recipe partial.
verifyfunctionReplay the recipe at path and, unless dry_run, write the verification back into the file.
write_recipefunction

function

def load_ipython_extension(ip: Any) -> None

%load_ext finprint.

Source: packages/finprint/src/finprint/__init__.py, line 54

function

def unload_ipython_extension(ip: Any) -> None

Source: packages/finprint/src/finprint/__init__.py, line 59

NameTypeValueDescription
__version__'0.2.0'