1 min lesson
Producers, consumers and the data contract
Use "Producers, consumers and the data contract" to explain each part and the role it plays.
Step 1 of 2
Producers, consumers and the data contractwho owns correctness when data crosses teams
Ownership gets murky once data crosses team lines - analytics, accounting and engineering all touch the same numbers. A clean mental model is producers vs. consumers. When finance runs its close on platform data, finance is the consumer; the partner teams emitting the upstream tables are the producers. The thing that makes the relationship work is an explicit data contract negotiated with those producers, covering three dimensions.
- Completeness
- Every row that should be there is there - no silently dropped partitions feeding the close
- Accuracy
- The values are correct, not just present - the producer asserts on content, not only on the job running
- Timeliness
- The data lands inside the window the consumer needs it (a close has a hard calendar deadline)
Snapshot-and-diff backs the contract: take snapshots as data changes and run reconciliations against them, so a breach is caught against a known-good baseline rather than discovered at close.
A finance-engineering team framed governance for the close exactly this way - producers, consumers and a contract - while being honest that the snapshot-and-diff reconciliation around it is still maturing.
"Having really clear understanding of the kind of... the contract, the data contract around completeness and accuracy, and timeliness."