Data contracts: promising, mostly aspirational

Data contracts have been “about to go mainstream” for four years now, which is a long time to be almost arriving. I’ve read the manifestos, tried a lightweight version in practice, and landed somewhere annoyingly moderate.

The diagnosis behind contracts is completely correct. Most pipeline breakage is an upstream team changing something they didn’t know anyone depended on. The app team renames a column in their own database, which is their database, and three dashboards die downstream. Nobody did anything wrong by their own lights. The failure lives in the gap between teams.

Contracts propose making the dependency explicit: schema, semantics, SLAs, agreed and enforced, so a breaking change fails the producer’s CI, not the consumer’s Tuesday morning.

Here’s where reality intrudes. A contract is only as strong as the producer’s incentive to honor it, and analytics teams are usually the least powerful party in that negotiation. When the app team’s deadline collides with your contract, guess which one bends. Tooling can’t fix an incentive problem. Half the “data contract platforms” I’ve evaluated are schema registries wearing an aspirational trench coat.

What actually worked for us was a de-escalated version I’d hesitate to call contracts at all. Staging-layer schema tests that catch drift the moment it lands, so the blast radius is one red build instead of broken marts. A short doc per critical source naming which fields we depend on and what “broken” means, which upstream teams cheerfully signed because nothing enforced it. And, honestly the highest-value piece, a Slack channel with the two engineers who own the noisiest sources. Most incidents got prevented by a human typing “heads up, we’re changing X next sprint.”

Detection on our side, plus relationships, plus written expectations. Contracts minus the enforcement fantasy.

If you have the organizational gravity to make real producer-side enforcement stick — dedicated platform team, leadership that will spend political capital on it — do it, sincerely. The rest of us should get the 80% that doesn’t require restructuring the company: test at your boundary, write down your dependencies, and know the humans on the other side of them.