LLMs in the pipeline: where they actually help

Every data conference for two years has featured a text-to-SQL demo, and every working data person in the audience has thought the same thing: my problem was never writing the SQL. Writing SQL is the fun part. My problem is that the marketing platform spells “United States” four ways and one of them is “USA “ with a trailing space.

So here’s my running list of where LLM-shaped tools have genuinely earned a place in pipelines I’ve worked on or watched closely, and where they haven’t.

Earned it: classification and extraction on messy text inputs. Support tickets into categories, free-text merchant names normalized, vendor descriptions mapped to a product taxonomy. This used to be regex graveyards and quarterly-retrained classifiers that everyone feared touching. A well-prompted small model with a validation layer does it cheaper than the maintenance burden it replaces. The validation layer is not optional; treat model output like any other untrusted source, because that’s what it is.

Earned it: entity resolution assist. Not deciding matches, but scoring candidate pairs for human review. “Is ‘Acme Corp’ the same as ‘ACME Corporation (formerly AcmeCo)’” is exactly the fuzzy judgment these things are decent at, and the human stays in the loop for the merge itself.

Earned it, embarrassingly: explaining legacy SQL. Handed a 400-line stored procedure written by someone who left in 2021, an LLM’s first-pass summary is a real head start, even when it’s wrong in places. Wrong-but-orienting beats a blank stare.

Not earned: text-to-SQL for stakeholders. The demo works; the org chart doesn’t. When the generated query is subtly wrong, who notices? The stakeholder can’t and the engineer wasn’t in the loop. That’s the dashboard-accuracy problem with extra steps and better marketing.

Not earned: anything where the model output feeds decisions without a validation gate. I hold this line even for the use cases I like.

The pattern across the list is almost boring once you see it. LLMs are useful in pipelines exactly where inputs are unstructured and judgment is fuzzy, which is precisely where traditional pipelines were always weakest. They’re an ingestion technology, not an analysis technology. The demos have it backwards.