Resolve Only What Changed
Rerunning matching on all your data is wasteful, and it loses the lineage of every record against its ID. Zingg Enterprise matches new, updated and deleted records against the clusters it has already resolved, and keeps the identity graph current.
Records already resolved are not matched again.
A new record that matches an existing entity joins its ZINGG_ID. Nothing else is touched.
Stop Rerunning Everything
Data changes every day. Matching all of it again for every change costs time and compute, and the IDs your systems store stop meaning the same thing.
| Rerun everything | Zingg incremental flow | |
|---|---|---|
| Records matched each run | All of them | Only new, updated and deleted records |
| Entity IDs | Rebuilt with every run | Persistent ZINGG_IDs that matched records keep |
| Lineage | Lost between runs | Every record stays traceable to its ZINGG_ID |
| Matches you approved | Need checking again | Kept: your feedback is never overridden |
| Cluster changes | Work out what moved yourself | Assign, merge and unmerge happen automatically |
Every Kind Of Change, Handled For You
Pass in what changed since the last run. Zingg finds where each record belongs in the identity graph and updates it.
New Records
A new record that matches an existing entity joins its ZINGG_ID. One with no match starts a new entity with a new, persistent ZINGG_ID.
Updated Records
If an update makes a record a better match for another cluster, Zingg reassigns it. Clusters merge or split as the new signals require.
Deleted Records
Pass the records to delete and Zingg takes them out of the identity graph. The rest of each cluster keeps its ZINGG_ID.
Approved Matches Stay
Feedback you gave on earlier matches is respected, so an incremental run never overrides records you approved.
Train Once, Then Run On Every Change
Train And Match Once
Label pairs, train the model and run a first match. Every resolved entity gets its ZINGG_ID.
Point At The Changes
Hand the incremental arguments your trained setup and the changed data: new and updated records, and the ones to delete.
Run runIncremental On Your Schedule
From Python or the command line, as often as your data changes. Incremental docs
# args: the EArguments you trained and matched with
changes = EPipe("changes", "delta")
deleted = EPipe("deleted", "delta")
incrArgs = IncrementalArguments()
incrArgs.setParentArgs(args)
incrArgs.setIncrementalData(changes)
incrArgs.setDeletedData(deleted)
phase = [ClientOptions.PHASE, "runIncremental"]
EZingg(incrArgs, ClientOptions(phase)).initAndExecute()Run It On The Platform You Already Have
Incremental runs execute as a native workload on your Spark cluster or in Snowflake, so changed records never leave your platform.
Incremental Flow Questions
What is the incremental flow?
A Zingg Enterprise feature that keeps the identity graph current. Instead of rerunning matching on all your data, each run matches only new, updated and deleted records against the clusters Zingg has already resolved.
Which records does an incremental run process?
Only the changes you pass in: new and updated records, and the records you want deleted. Records that are already resolved are not matched again.
Do ZINGG_IDs change between incremental runs?
No. A new or updated record that matches an existing entity joins its ZINGG_ID. Only a record with no match gets a new, persistent ZINGG_ID.
What happens when an updated record fits a different cluster?
Zingg reassigns it. Cluster assignment, merge and unmerge happen automatically: an update that links two clusters merges them under one surviving ZINGG_ID, and a record that no longer fits its cluster moves out to a better match or an ID of its own.
Can incremental runs handle deleted records?
Yes. Pass the deleted records to the incremental run and Zingg takes them out of the identity graph, while the rest of each cluster keeps its ZINGG_ID.
Will an incremental run override matches I approved?
No. Zingg Enterprise takes human feedback on previously matched data into account, so approved records are not overridden.
How do I run it?
Train a model once, then point an incremental config at the changed data and run the runIncremental phase, from the command line or from Python, as often as your data changes.
Which edition has the incremental flow?
Zingg Enterprise, on Databricks, Snowflake, Microsoft Fabric, GCP or AWS.
Keep The Identity Graph Current
Incremental runs and persistent ZINGG_IDs come with Zingg Enterprise, on your own data platform.