What "done" means when the deliverable is a number

A software engineer finishes a feature and can show you. Click the button, the modal opens, the record saves. Everyone in the room can tell whether it worked, including the person who asked for it, including the person who has no idea how it was built.

I ship numbers. Nobody has ever looked at 4,182 and known.

That asymmetry is most of what makes data work hard to close out. A review meeting for a new dashboard ends with somebody saying “yeah, that looks about right,” which is a statement about their expectations, not about my pipeline. If the number lands inside the range they already believed, it passes. If it lands outside, they ask me to double-check, and often enough I find a genuine bug, which teaches the room exactly the wrong lesson: that eyeballing works.

Eyeballing works right up until the number is wrong in a plausible direction. Then it sails through and lives in a dashboard for a while.

The alternative is not complicated. Before building, write down what has to be true about the output for it to count as correct. Not “the revenue view is done” but the checkable conditions. The daily total reconciles to the billing system within a stated tolerance. The grain is one row per account per day, with no duplicates. The table is never more than six hours stale. No row has a null customer id. Segment totals sum to the unsegmented total.

None of that is clever, and that’s the first honest thing to say about acceptance criteria for data. They are mostly a written version of the checks you would have done by eye anyway, right before pasting the link into Slack. What writing them down buys you is that a habit becomes an artifact, and an artifact can run at 3am on a schedule. Habits cannot. Habits also get skipped on the Friday when three other things are on fire, which is statistically the day something breaks.

At my last job I moved that checking out of my own head and into the ELT as validation rules, so a failed check stopped a load instead of producing a quietly wrong table. Dashboard accuracy went from roughly 88% to 98% over the following months and monthly report rework dropped about 30%. The incident that finally convinced me to do it is written up here, and I’d rather you learn it that way than mine.

The checks are the easy half. The hard part is agreeing on them with the person who asked, and that conversation is the one everybody skips.

I once delivered a churn number that was completely correct and completely useless, because I had counted trial accounts that expired without converting as churn. The person who requested it meant paying customers only. She had never said so, because to her it was too obvious to say, and I had never asked, because to me the definition looked settled. The fix was four words in a WHERE clause. Agreeing on which definition was right took considerably longer than building the model did, and involved a second team who had a third answer.

That is what acceptance criteria are actually for. The checks catch drift later. The act of writing them catches disagreement now, while disagreement is still cheap and nobody has presented the number to a VP yet.

Write them in the requester’s language, then translate. “Every paying customer appears exactly once” is a sentence a growth lead can argue with. unique on (account_id, snapshot_date) where plan_type != 'trial' is not, and sending it produces a thumbs-up emoji that means nothing. Send the first version, implement the second, and keep them next to each other in the model’s documentation so the next person can see both.

You’ll also get stakeholders who won’t engage at all, who say some version of “you’re the data person, you tell me.” That’s fine, and honestly it’s most of them. Write your best guess at the definition, send it, and say you’ll build it that way unless you hear otherwise. Silence becomes consent, on the record, with a timestamp. It feels slightly bureaucratic. It has saved me from rebuilding things more times than my ego wants to admit.

I used to think this was paperwork. My first year working with data I would have told you that writing acceptance criteria for a dbt model was a way of looking productive in front of a project manager, and that real work was the SQL. The SQL was never the risky part. The risky part was two people using the same word for different things and both being satisfied with the meeting.

Nobody has ever thanked me for a validation rule. They thank you for the number being right, which they had assumed it always was.