The backfill was meant to take twenty minutes. A data engineer had written the script on a Friday afternoon, run it against a small sample, and felt good about it. The sample looked right, the logic was tidy, and the stakeholder was waiting on a corrected report. So the script went to production, and by the time the first dashboard refreshed, a month of customer records had been overwritten with values from the wrong time zone.
Nobody on that team was careless. They were busy, trusted each other, and had done this kind of operation many times. What they lacked was a short list they read before the step that could not be taken back. After the clean-up weekend, they wrote one. This article shares the eight lines it ended up with, because a pre-flight checklist for data engineering is one of the cheapest habits a team can adopt.
Why a checklist and not just care
Data work has a particular trap. Most steps are reversible: you rerun a job, rebuild a model, reload a file. A few steps are not, such as dropping a table, overwriting a partition, or rewriting history in a warehouse without a copy. Because the reversible steps are so forgiving, people develop a relaxed rhythm, and that rhythm carries straight into the one step that deserves a pause.
A checklist interrupts the rhythm at exactly the right moment. It does not replace skill. It simply makes the engineer say out loud what they already know, in the same order, before pressing enter. The team in this story kept the list to a single screen and applied it to the launch of the risky step only, not to the whole project.
The eight checks
**One: say what is about to change, in a sentence.** The first line on the list is a plain statement: “This script rewrites the amount column for every row in this table between these two dates.” If the sentence is hard to write, the change is not understood well enough to run. The team found that half of their near misses started with a vague description such as “fix the totals”.
**Two: confirm which environment the terminal is pointed at.** That looks too basic to list, yet it causes more slips than anything else here. Connection strings look alike, and a shell left open from the morning is still connected to production. The engineer reads the host name from the prompt rather than trusting memory.
**Three: take a copy you have actually located.** Everyone says there is a backup. The check is stronger: open the copy, query a few rows, and confirm it is recent enough to be useful. A snapshot that exists but cannot be restored in time is only a comforting rumour.
**Four: count what you expect to touch.** Before the change runs, run the same filter as a read-only query and note how many rows it selects. Compare that figure with what the engineer expected. If the expectation was a few thousand and the filter returns the whole table, the script has just been stopped by arithmetic.
**Five: check who else reads this table.** Downstream jobs, reports and shared notebooks may rely on the current shape. A quick look at lineage or at recent queries shows whether a dashboard will break mid-run. This line is also where a second person gets pulled in, because someone else often owns the consumer.
**Six: choose a window with nobody waiting on the data.** Not an off-peak argument, just a courtesy check: is a scheduled load, a finance close or a customer export about to read this table? If so, the change waits or the other job is paused first.
**Seven: name the person on call and the way back.** The engineer writes down who is reachable for the next few hours and which command or restore path undoes the change. If the way back is “we will figure it out”, that answer goes on the page honestly, and usually the launch is postponed.
**Eight: tick the list out loud with a colleague listening.** The final line is social. A short message or a quick call, with the answers spoken rather than silently assumed. The team found this was the line that caught the most mistakes, because saying “production, not staging” to another person makes the error audible.
What changed after the list arrived
The team did not transform overnight. For the first few weeks, a couple of engineers grumbled that the list slowed them down. Then a junior engineer, preparing to drop an old staging table, reached line two and realised the terminal was connected to the wrong database. The table in question turned out to hold live reference data. Nobody mentioned the grumbling again.
Over the following months a few patterns emerged. Hand-offs improved, because anyone could pick up the list and know what the team considered essential. Reviews got shorter, because a reviewer could ask “did you run the list?” instead of re-deriving every concern. And the list itself shrank, because items that never caught anything were quietly dropped.
How to write your own list
Start from your own near misses rather than from a template. Gather the last few incidents where a data change went wrong and ask which single question, asked just before the step, would have caught each one. Those questions are your list.
Keep the wording concrete. “Confirm the copy is restorable” beats “ensure data protection”. Keep the order stable, because reading in the same sequence turns the list into a rhythm instead of a chore. And keep the list about the launch, not the entire job. Design reviews, testing and data-quality rules all happen earlier and elsewhere. The checklist is for the last minute before the point of no return.
A useful trick is to attach the list to the tools people already open. A pull request template, a runbook header or a pinned message in the team channel all work. If people have to hunt for the list, they will skip it on the day it matters.
Where the habit comes from
Checklists before irreversible steps are not a data invention. Pilots, surgeons and kitchen crews all use them, and the principle travels well because it targets human attention rather than technical skill. For a look at how a very different technical community explains careful step-by-step procedure, CryptoBlockCoins publishes a blog about cryptocurrency and blockchain that includes coin, tutorial and daily-log sections. Its tutorials show how ordered, explicit steps serve readers even when the subject leaves little room for error.
A few honest limits
A checklist will not catch a flawed idea. If the logic of the backfill is wrong, a perfectly ticked list will carry that wrong logic into production with great discipline. Testing and review remain essential, and the list assumes they happened already.
Lists can also decay into ritual. If people tick items without looking, the list is worse than useless, because it creates false confidence. The cure is to rotate who reads it aloud and to make at least one item require an action, such as opening the backup and running a query against it.
Finally, not every change deserves the full list. Rerunning a daily aggregate that rebuilds itself does not need eight lines. Reserve the ritual for steps where undoing the damage would cost days.
More from this site
To round out the wider data practice, two earlier posts here sit nicely beside this one. The first looks at analytics that stay live while pipelines wobble. The second explores running data operations across several clouds. Both describe the surrounding practices that make a short launch list possible in the first place.
The one-page version
If you take away only one thing, take this: ahead of any step you cannot reverse, stop, read one short list top to bottom, and speak the answers to a colleague. Eight lines were enough for the team in this story. Your list might have six or ten. What matters is that it exists, that it is short, and that it gets read on the Friday afternoon when everyone is tired and the script looks perfect.