Without a unique identifier, every record could match any other. Zingg takes care of scale for you: as you label pairs, it learns which records are worth comparing, and matching runs on millions of records on the platform you already have.
Comparisons grow with the square of your records: ten times the records means a hundred times the work.
The pairs you label to train matching also train the blocking model, automatically. There are no blocking keys to design, tune or maintain.
john doe goes j, then new, and is compared with johnh d oe and jon doe only.
Scale is built in: Zingg learns the blocking model while you label, so there is nothing to set up.
Every Yes and No you give teaches Zingg which records belong together. From the same answers it builds the blocking model, so scale needs no extra work from you.
At every node of the blocking tree, Zingg picks the field and function that keep your labeled matches together. Each branch can split on a different combination, so near-similar records land in the same block and the cartesian join breaks.
Run time depends on the number of records, the number of fields compared and how many duplicates there really are. These runs give a sense of the hardware you need.
Up to a few million records, a single machine in Spark local mode is often enough. Beyond that, Zingg runs as a native workload on your Spark cluster or in Snowflake, and scales with it.





Run verifyBlocking before a large match to see the record count per block and samples from each, and add labels if blocks are too big.
Incremental runs match new and updated records against existing clusters instead of reprocessing all your data.
Incremental flowZingg has been run on tens of millions of records. In Zingg's own benchmarks, 9 million records with three fields took 45 minutes on a 96-core machine, and a user reported 80 million records with 8 to 10 fields in under 2 hours on a driver and 8 workers.
Not for smaller data. With up to a few million records it can be easier to run Zingg on a single machine in Spark local mode. Larger data runs on a Spark cluster on Databricks, Microsoft Fabric, GCP or AWS, or natively in Snowflake.
It learns a blocking model, a tree of fields and functions, from the pairs you label. Each node picks the field and function that keep your labeled matches together, different branches can split on different fields, and only records that end up in the same block are compared.
No. The blocking model is learned automatically from the same pairs you label to train matching, so there are no blocking keys to write or tune.
The number of records, the number of fields being compared and the actual number of duplicates. Spark partitions matter too: keep numPartitions at 20 to 30 times the number of cores.
Yes. In Zingg Enterprise, the verifyBlocking phase shows the record count per block and sample records from each, so you can see whether you need more labels before matching everything.
Start with open source today, or talk to us about running Zingg Enterprise on your data.