Writing

Notes from the pipeline

Things I've learned building data platforms — what broke, what worked, and what the hype cycle gets wrong. New posts most months.

  1. The dashboard was wrong for six weeks and nobody noticed

    A join quietly started dropping rows and the revenue dashboard kept looking plausible. What I changed so it never happens silently again.

  2. Semantic models are the most underrated work in analytics

    Nobody puts "built a measure library" on a conference slide, but it took our reporting cycle from two days to four hours.

  3. Iceberg won. Now what?

    Everyone agrees on the table format now. That settles much less than people think.

  4. What consolidating 8 source systems taught me about ELT design

    Lessons from pulling product, marketing, and customer data into one Snowflake analytics layer without breaking anything downstream.

  5. dbt tests are unit tests for your beliefs

    You think order_id is unique. The test is how you find out when you stop being right.

  6. DuckDB is my new scratchpad

    Not everything needs the warehouse. A local engine that eats parquet has quietly changed how I debug and prototype.

  7. Cutting our Snowflake bill without touching a single dashboard

    Full-refresh models were rebuilding the world nightly. Making them incremental cut warehouse compute hard, and nobody upstairs noticed a thing. That was the point.

  8. The anomaly detector that cried wolf

    My first risk model was technically excellent and practically ignored. Fixing it had nothing to do with the model.

  9. Data contracts: promising, mostly aspirational

    The idea is right. The org chart is the hard part. Notes from trying a lightweight version in real life.

  10. BigQuery optimization is mostly about not being clever

    How warehouse workloads got ~45% faster with partition pruning, honest data types, and the removal of my own smart ideas.

  11. Nobody reads your data docs. Write them anyway.

    Documentation and lineage felt like chores until self-service went up 60% and my interrupt load collapsed. Turns out docs are read at exactly one moment.

  12. LLMs in the pipeline: where they actually help

    Ignore the text-to-SQL demos. The wins are smaller, weirder, and mostly about unstructured inputs and terrible source data.

  13. From business analytics to data engineering: what actually transferred

    I came in through the analytics door, not the CS door. Less of a handicap than the discourse suggests, with a couple of real gaps to close.

  14. Kicking the tires on Microsoft Fabric

    A Snowflake-and-dbt person spends real evenings with Fabric and comes back with mixed, specific feelings.

  15. Boring pipelines are a feature

    The most senior thing a data pipeline can do is nothing interesting. On idempotency, retries, and the luxury of not being needed at 3 a.m.