A widely circulated essay by data consultant Chad Sanderson argues that most enterprise lineage tools map the warehouse rather than the application code that actually produced the data — a distinction he says separates decorative diagrams from something an auditor can actually use.
Every data team has lived the same interrogation. A regulator, an internal auditor, or a model risk reviewer asks how a number was produced, which inputs fed it, and where those inputs originated — and the honest answer takes a week of interviews, some archaeology through old repositories, and a spreadsheet three people vouch for and one person quietly distrusts. That ritual is the opening of “The Shift Left Manifesto – v2,” an essay published on dataproducts.substack.com on July 24, 2026, an update to a piece the author says is “often cited as a way to introduce data concepts into software engineering.” The essay carries the byline Chad Sanderson and is dated July 9, 2026 in a copy circulating alongside it — a discrepancy with the Substack page’s own July 24 date and missing byline that this outlet could not independently resolve. The argument underneath both versions is unchanged: most lineage tooling starts at the warehouse, and by the time data lands there, according to the piece, “the interesting decisions about it have already been made, somewhere else, by code nobody mapped.”
borrowing from devops
Sanderson’s framing leans on a pattern he says has already played out twice in software engineering. DevOps, he writes, took deployment and infrastructure “which used to be a downstream handoff to a separate operations group” and moved them into the development workflow itself, throwing off version control and CI pipelines as a byproduct. DevSecOps did the same to security, and feature-flag management did it to rollout control. “Each time, a function that lived downstream moved into the place where code is written, and a durable record fell out as a side effect,” the essay argues. For data, Sanderson says the discipline that needs to move is provenance, and the byproduct is lineage — a reframing that recasts a decade of data-lake architecture, in his telling, as a reactive discipline built to reconstruct meaning “after the fact from fragments produced by code it never saw.”
warehouse lineage’s blind spot
The essay’s sharpest claim is that “database lineage is not data lineage.” Sanderson distinguishes storage lineage (table-to-table) and transformation lineage (SQL and transform-job parsing) from what he calls code-level lineage — tracing a field back to the service, function, and business rule that produced it before it ever reaches a data platform. “A tool that reads only the database is reading the last chapter and trying to reconstruct the plot,” he writes. That’s a direct challenge to how incumbent governance vendors describe their own category: Databricks lists “end-to-end lineage visualization” among four foundational capabilities of a governance platform, calling it “a non-negotiable requirement in complex data environments, where a single incorrect transformation can propagate errors across hundreds of downstream reports and AI models” — precisely the warehouse-anchored view Sanderson says starts too late. A similar epistemic argument has already reshaped an adjacent field: Wiz’s guide to vulnerability prioritization makes the case that raw CVSS severity scores are noise without environmental context — reachability, exposed permissions, adjacency to sensitive data — the same move from static signal to contextual, source-level evidence that Sanderson is prescribing for data provenance.
the streaming crowd agrees, sort of
“Shift-left” has become a crowded banner. CDOTrends, covering Confluent’s fifth annual Data Streaming Report — a survey of 4,625 IT leaders — reported that half of respondents say streaming pays back at least five times what they put in, with 90% crediting streaming for accelerating AI adoption and 94% expecting it to amplify AI spending. CDOTrends flagged the obvious caveat: “Confluent sells the platforms it’s measuring, so keep a thumb on the scale.” The overlap with Sanderson’s essay is instructive rather than coincidental — both frame shift-left as moving cleanup and governance work upstream of storage, though Confluent’s version is about data freshness for AI pipelines, while Sanderson’s is about provenance for audit and change-management questions. Neither essay nor report discloses pricing, adoption figures, or named customers for a code-level lineage product; the piece stays at the level of argument, not a vendor pitch, and names no company Sanderson is building.
what to watch
Simplilearn’s 2026 trends survey lists “analytics moving from dashboards to governed metrics and faster decisions” alongside AI governance and identity-first security as capabilities already showing up in hiring and budgets — the broader current Sanderson’s essay is trying to ride. Whether code-level lineage becomes a purchasable category, rather than a consulting thesis, is the open question; the essay itself offers no product, pricing, or customer count, only Sanderson’s claim to have “spent the last few years working with enterprises trying to close that gap.” The three questions he says the approach answers — what breaks if I change this, what will I get wrong, what already exists that I can reuse — are the kind of claims that are easy to state and hard to verify without a deployed tool in front of a skeptical engineering org. Until then, it’s a manifesto, not a market.
Database lineage is not data lineage. Or more precisely, it is one layer of data lineage, and usually the layer that starts too late. The provenance of a value, the real answer to where it came from and what it means, mostly does not live in the database. It lives in the code that produced it.
