Data teams live on scheduled jobs: backfill runners, retention purges, partition repairs, replica failovers. Each one was written carefully once, and each one now acts on production data with nobody watching. AutomationOps is the discipline of running that background fleet the way you would run software you ship: inventoried, tiered by risk, guarded, and owned until the day it retires. It is not another pipeline framework; it is the maintenance manual for everything already running around your pipelines. The reference definition lives in the Ops Atlas entry on AutomationOps and its five core controls, and this post turns that material into something a data platform team can actually run: a twelve-point audit, scored in an afternoon, that tells you which of your scheduled actions would behave during a bad week and which are one malformed input away from a very long incident review.

How to Run This AutomationOps Audit

Gather the people who own the schedulers, the warehouse, and the on-call rota. List every recurring operational action you can find — cron entries, orchestrator tasks, event-triggered handlers — and walk the twelve checks below against each one. Score a point only where the check passes on paper and in practice; a policy nobody has exercised counts as a fail, politely.

Expect the first pass to be humbling. Most teams discover actions nobody remembers approving, credentials shared across a decade of scripts, and at least one job whose failure would be discovered only when a downstream table goes quiet. That discovery is the audit working. Bring three artefacts to the session: a scheduler export covering the last ninety days, the list of identities your scripts authenticate as, and the incident tickets where an automated job appears in the timeline. Cross-referencing those three turns opinions into evidence in minutes. The point is not blame; it is a map of where a routine job turn could become a data-loss event.

The Twelve Checks

1. An inventory exists. Every recurring operational action appears in a list with a name and a purpose. Count the ones missing from it; that unregistered remainder is the estate’s blind spot, and it only grows while nobody measures it. A spreadsheet beats nothing, and a versioned file beats a spreadsheet.
2. Each entry has a named owner. Not a team alias — a person, plus a review date. Ownership coverage above ninety percent signals an estate somebody actually tends; below that, you are operating scripts with no adult present. When an owner leaves, the entry is reassigned that week, not eventually.
3. Every action carries a risk tier. Read-only work sits at the bottom; irreversible, data-destroying actions sit at the top and demand a second pair of human eyes before they run, plus soft deletion and staged execution where data is involved. Middle tiers earn progressively heavier guards between those poles.
4. A dry-run mode exists — a real one. Some configuration tools skip command probes in their check modes unless you flag them explicitly, so a “passed” preview can exercise far less than it appears to. Verify what the dry run actually evaluates before trusting it with a fleet-wide change.
5. Empty target lists are refused. Google’s post-mortem on Diskerase — a drive-cleanup tool that read a blank host list as “the whole fleet” and deleted accordingly — became a standing lesson: a selection matching nothing must halt the run, never widen it. The guard is one line of code; the habit is priceless.
6. Broad changes run in batches. A canary wave first, then staged groups, with a ceiling on how many failures a wave may absorb before the whole run stops. Configuration tooling expresses this through serial execution and failure-percentage settings; hand-rolled scripts deserve the same discipline in their loops.
7. A failure budget halts the run. Beyond N failed targets the automation stops on its own rather than grinding through a broken fleet making things worse. Pick N per action, write the reasoning next to it, and revisit the number when the fleet grows.
8. Halting is possible twice over. One stop lever per automation, one master switch for the runner. For the riskier tiers the drill repeats on a six-month rhythm, and whoever holds the pager may pull either lever without seeking consent first. A stop nobody has ever pulled is a theory.
9. Identities are individual. Each automation authenticates as itself, with its own scope. A shared god credential turns every misfire into an unattributable one and makes revocation impossible without stopping the world. Per-job identities also make your logs tell the truth about who did what.
10. Repeats are written as state declarations. “Ensure running”, not “start again”; “ensure present”, not “create”. A job phrased this way can run twice in a row without harm, which is the difference between a retry and a disaster. Actions that cannot be repeated safely — charges, sends, creations — must carry caller-supplied request identifiers recorded atomically, so a duplicate attempt collapses into the original.
11. Runs emit the four numbers that matter. Success rate; hours genuinely returned once maintenance is subtracted; how often a human had to take over; and how often the automation itself triggered an incident. If none of these exist per job, value is a story rather than a measurement.
12. Retirement is scheduled. Every entry has a review date, and letting a script go is treated as progress. Automation that outlives its explanation becomes governance debt with admin rights attached.

Two Shortcuts That Speed the Audit Up

First, start from the actions that touch data at rest rather than the ones that merely read it. Deletion, overwrite and migration jobs carry the tail risk; verifying them first buys the most safety per hour spent. Second, audit the newest scripts last — the oldest ones have had the most time to drift from the system they manage, and they usually fail the identity and review-date checks immediately.

One more shortcut: wherever a check fails, write the fix into the entry itself while the room is still together. An audit that ends with a queue of agreed changes beats a beautiful report nobody actioned.

What the Score Actually Means

Tally honestly — a check passes only if you could demonstrate it to a newcomer in ten minutes, using the live system rather than a slide.

• 0–4: the estate is running on goodwill. Start with the inventory and the empty-list guard; both are days of work and together they remove the worst failure modes on the board.
• 5–8: the bones are sound. Put the next effort into individual identities, failure budgets, and the first halting drill so the stop levers become real.
• 9–12: you are governing automation rather than merely surviving it. Keep the review dates honest and watch the induced-incident figure for slow drift.

From Audit to an AutomationOps Registry

The audit is a snapshot; the registry makes it continuous. Move each finding into the entry itself — tier, limits, identity, owner, review date — and the twelve checks become ten minutes of reading instead of an afternoon of archaeology. When a new job is proposed, it arrives with its tier and guards already drafted, the way a pull request arrives with tests. Data teams that have already tightened their pipeline practices will recognise the pattern immediately: the same rigour you apply to schema changes belongs to the scripts that clean up after them. The guides on keeping real-time analytics from breaking, real-world DataOps implementations and multi-cloud data operations cover the pipeline side; this audit covers the scripts living around it. Schedule the audit twice a year, hand the template to whoever inherits the platform next, and the estate stays explainable — which is the entire point.