Imagine a Monday report where weekly revenue for one region has fallen by a third. Before anyone opens a ticket, a quieter question matters more: which parts of that sentence do you actually know? A plain-language look at how data engineers reason about a suspicious number begins here, with the difference between what a table can confirm, what a team assumes about the upstream feed, and what is only a hunch.
Start with the claim, then split it into parts
“Revenue fell” feels like one fact. It is really a bundle of smaller statements, and each one has its own level of support.
- The dashboard shows a lower total for that region. You can see this directly, so it is known.
- The dashboard reads from the same table as last week. That is an assumption until someone opens the query and checks.
- Customers in that region really spent less. This is a guess, since a broken join or a late file produces the same chart.
The exercise is simple, but it changes the order of work. Instead of arguing about why customers changed their behaviour, the team first finds out whether the data even covers the same period as before.
Why data work makes this easy to get wrong
Data pipelines have long chains: a source system, an extraction job, a staging area, transformations, a model and finally a chart. Each link is owned by somebody else, and each hands over a result that looks finished. The further downstream you stand, the more of the chain you accept on trust. Most “data incidents” are really cases where a downstream reader assumed something about an upstream step that had quietly changed.
Common quiet changes include a column renamed at the source, a time zone applied twice, a file that arrives late and gets skipped, or a filter someone added to a view for a different report. None of these shows up on the chart as an error. They show up as an odd number, and odd numbers invite stories.
A worked example, in three passes
Take the falling-revenue case and walk it through three passes, each producing a different list.
- Known list. Row counts for the region by day, taken straight from the warehouse. The count drops sharply on two specific days and recovers afterwards. That pattern is observed, not inferred.
- Assumed list. The nightly load ran on those days. The currency conversion table was current. Duplicates are removed in the same way as before. Each item goes on the list as something to confirm, with the check beside it: the scheduler history, the table’s last update time, the deduplication step.
- Guess list. A promotion ended, a competitor cut prices, or a payment partner had an outage. These are plausible, and they are all stories about the world rather than about the pipeline.
Working the assumed list first is cheaper than chasing guesses. In this example, the scheduler history shows the load for one of the low days finished with a warning and wrote only part of the file. One assumption is now a known fact, and the guess about customer behaviour has lost most of its footing.
Plain tests that move items between lists
You do not need sophisticated tooling to promote an assumption to a fact. A few humble checks do most of the work.
- Compare against the source of truth for the same slice. Count orders for one day in the operational system and in the warehouse. If the two differ, you have located the gap.
- Check freshness. Look at the latest timestamp in each table in the chain, not just the last one.
- Trace one row. Pick a single record you know exists and follow it from entry to chart. If it vanishes somewhere, that stage is under suspicion.
- Re-run a small slice by hand. If the manual result matches the dashboard, the logic is faithful and the inputs are the question; if not, the logic is.
Every check poses one narrow question and returns a yes or a no.
Writing it down so the label travels with the claim
When the finding goes to a stakeholder, the labels should go with it. A message that says “the dip is caused by a partial load” is stronger when it adds how you know: “the scheduler log shows a warning on those days, and the row counts match a half-written file.” A message that says “likely a promotion ending” should say plainly that it is a guess awaiting a check against the marketing calendar.
Teams that keep this habit get fewer reversals. People stop quoting a number from a meeting as settled fact, because the notes show which parts were still open.
Related reading on this site: one article looks at live analytics that survive broken pipelines, and another at data work spread over multiple cloud providers, where each extra handoff gives an assumption one more place to hide.
A note for people learning the craft
Learners often practise on tidy datasets where every assumption holds. Real pipelines punish that habit. One way to build better instincts is to study incident write-ups and ask, line by line, which statements were verified at the time. For structured material, DevOpsSchool offers an online LMS aimed at software engineers, where DataOps sits alongside DevOps, MLOps and site reliability topics, with tutorials, cheatsheets and quizzes among its study resources.
The habit in one paragraph
The next time a figure looks off, hold back from explaining it. Write three short lists: what you observed, what you are taking for granted, and what you only suspect. Test the middle list before the last one. Share the result with the labels still attached. The habit takes a few minutes, and it keeps a data team from confidently explaining an outage that was really a half-loaded file.