Survivorship & golden records
Matched pairs become clusters, clusters become one golden record each — via explicit survivorship rules — plus the crosswalk that ties history together.
Matching yields pairs: A≈B, B≈C. Entities are clusters: {A, B, C} — note A and C were never directly compared; they're joined transitively. The standard tool is union-find (a few lines of code, effectively linear). Transitivity is powerful and dangerous in equal measure: one bad edge (a shared office phone) can chain two real families into one blob-entity. Production systems cap cluster sizes and flag suspiciously large ones for review; you'll see why in the lab when you inspect your biggest cluster.
Survivorship: one record from many
A cluster of three records has three names, two emails, two phones. The golden record needs one of each. Survivorship rules decide, field by field — and the crucial property is that they're explicit and written down, because 'which value wins' is a business decision wearing a technical costume:
- Most trusted source wins — Harbor Lane trusts web signup (typed by the customer) over POS (typed by a cashier at rush hour) for name and email.
- Most recent wins — for volatile fields like address: the newest verified value.
- Most complete wins — a non-null beats a null;
elena.diaz12@example.combeats an empty string regardless of source. - Never invent — golden records select and combine; they don't synthesize. If no source has a phone, golden has no phone.
The crosswalk: the table that makes it all usable
source_ref customer_key
W0042 -> 481
P0107 -> 481 <- same human, both refs resolve to golden #481
W0119 -> 482The golden dimension answers 'who is customer 481?' The crosswalk answers the question every fact table asks: 'this order says P0107 — whose is it?' Stamp customer_key onto fct_order_lines via the crosswalk and, for the first time, COUNT(DISTINCT customer_key) is Harbor Lane's true customer count and repeat-purchase analysis sees whole humans. Keep source refs on the fact forever — resolution improves over time, and the crosswalk re-stamp is how facts follow.
New records arrive daily; matches that were gray become confident; rules improve. Golden keys must be stable across runs (customer 481 stays 481 when a new fragment joins the cluster) or every downstream reference breaks. This is genuinely hard to get right incrementally — it's a core piece of what Edova's Meld productizes, alongside review queues and per-field survivorship policies, over Snowflake, Databricks, Postgres, and BigQuery.