The premise
Cleaning the data first is a sequencing decision.
And in a live operation it is usually the wrong sequence. Not because the data is fine — it isn’t — but because of what the order does to everything queued behind it.
Every operation of any size runs on data that is a little wrong. A master record that was correct when it was created and has drifted since. A field two teams fill in differently because nobody ever agreed what it meant. A number that is right in one system and a day stale in the next. Nothing is broken. This is simply what a large, interconnected system looks like from the inside, and it is the normal condition rather than the exception.
The instinct is to fix all of it before building anything on top. That instinct is not wrong about the data. It is wrong about the order of work. It places every unit of value behind the least controllable dependency in the business, and it assumes a finish line that a trading company never reaches: while you clean, the operation keeps running, and keeps producing new inconsistency at roughly the rate you are removing the old.
We have watched data-cleanup programs run for years and close without ever delivering the improvement they were funded to unlock. Rarely because the work was poor. Usually because the plan required a steady state that no operating company holds still long enough to reach.
From the field
On one program, a single inconsistency in long-standing data took about eight months to track down. Two of those months were real, palpable delay — a signed-off improvement sitting still, waiting on a fix nobody could yet name.
The design brief
Four behaviours, designed in from the start.
Clean the inputs where you reasonably can, and put guardrails around them. Then design for the data you can realistically have rather than the data you wish you had — which means a solution resilient enough to do four things.
Tolerate what doesn’t matter.
Most inconsistency is noise against the decision actually being made. Two systems disagreeing on a description field does not change a replenishment call. Decide deliberately which variances the answer is insensitive to, and let those through untouched. A system that halts on every imperfection is not rigorous. It is unusable, and people route around it within a fortnight.
Detect what does.
For the variances that move the answer, detection has to be continuous rather than periodic. Reconciliation between two systems stops being an exercise someone runs at month end and becomes a property the system holds — checked every time the data moves, not every time somebody remembers to look.
Recover where it can.
A large share of what gets detected can be resolved without a person: reconciled against a second source, inferred from a pattern the system has seen a thousand times, or corrected against a rule the business already applies by hand every day. That long tail is exactly where machines outperform people, because the work is high-volume, low-judgement, and relentless.
Escalate what needs a human.
What survives the first three is the head of the distribution: fewer items, higher stakes, real judgement. Route it to a person with the context attached and the decision already framed. This is not the system failing. It is the system working — it is how scarce attention gets spent where it is worth the most.
“The right design decision is to acknowledge this. Not to demand that the data is corrected first and then you do something, but to design a system where a minor inconsistency either does not affect the result, or is corrected automatically, or is detected automatically.”
Then, and only then, improve the inputs — where the business value justifies it. That last clause is the whole difference between a backlog and a program.
What it changes
This is an operating-model decision, not a technical one.
Accepting imperfect inputs as a design premise changes what an organization funds, what it measures, and where it puts its people. Four consequences follow almost immediately.
The interesting question was never how clean the data is. It is how much imperfection your operating model can absorb before someone has to stop and fix something by hand.
Data quality stops being a project.
It becomes a running property of the system, with no completion date — and none needed. The governing question moves from “is the data clean?” to “is the system still producing the right answer, and how quickly would we know if it stopped?”
The escalation queue becomes a management artifact.
What the system hands to humans each week is the clearest available statement of where your operation is genuinely ambiguous. Read it as a diagnostic rather than a defect list. Its shape tells you which handoff to redesign next, and its trend tells you whether the last redesign worked.
Cleanup becomes a ranked, funded backlog.
Some inputs are worth fixing at source. Most are not. Detection data tells you which is which, with a number attached — how often this field is wrong, and what it costs when it is. That is a business case rather than a hygiene argument, and it survives contact with a CFO.
Handoffs get designed, not assumed.
Much of what gets called a data problem is a coordination problem wearing a data problem’s clothes: work passed over the fence with no agreement on which fields travel with it, in what shape, or who owns them on the other side. Fix the handoff and the data problem stops recurring. Fix the data and it comes back next quarter.
How we apply it
What this looks like inside an engagement.
This is not a position we hold in the abstract. It decides which cases we take first, what ships in week one, and what we keep watching after go-live.
Cases are scored on the data you already have.
In the diagnostic, every candidate case is ranked on value and on how fast we can ship it — and speed is largely a question of what the available data can already support. A case that needs an eighteen-month cleanup before it can start is not a fast case, whatever its headline value.
The four behaviours ship in the first build.
Tolerance thresholds, detection, automatic recovery, and escalation rules are in the first version your team sees, not in a later hardening phase. Retrofitting them is how a promising pilot quietly becomes unusable in month three.
Escalation is the control model, not an exception path.
Full auto-write for low-risk actions, review-and-approve for the rest, read-only display where you want it. You decide per workflow, and every action that matters waits for a human with a full audit trail behind it.
The escalation rate is a measured number.
If the system escalates more this month than last, something upstream moved. That is a signal we act on, and one of the reasons we keep monitoring after go-live instead of handing over and leaving. Most of the risk to an operational AI system shows up after it is live.
Where this came from
A conversation about deciding quality at the design stage.
Mike Fedorov worked through this thinking with Dianna Deeney on Quality During Design, her podcast for engineers and designers who would rather decide quality at the design stage than inspect for it afterwards. The stretch on designing for imperfect data starts around the thirteen-minute mark.
It is a fitting place for the argument to land. Designing for imperfect data is the same move Dianna makes about quality generally: the decision that matters is taken early, in the design, not late, in the inspection.
Start with what you have
Curious what your current data could already support?
A one-to-two-week diagnostic gives you a ranked shortlist of the operational cases worth solving, scored on value and on how fast they can ship with the data you have today.