Nobody on the platform team at Merwick Foods asked to become security engineers. They maintained pipelines, kept the warehouse healthy, and fielded the occasional midnight page. Then the SOC queue started leaking onto their desks: service-account anomalies, odd bucket reads, alerts naming jobs they owned. Their director proposed a trial of AIOps for Security tooling — the emerging label covering ML scoring, behaviour analytics, and LLM assistants inside security work — and volunteered the data team to run it, on the theory that people who measure pipelines for a living would measure this properly. What follows is the trial as they actually ran it, failures included.

Week Zero: Writing Down What “Better” Means

The team’s first act was refusal — of the vendor’s success metric. The pitch deck celebrated “percent automated”, a number they recognised instantly as a vanity count: it measures how much the machine touches, never how often it is right. Instead they wrote two sentences on the whiteboard. First, a miss — a genuine threat closed as benign — is the worst possible outcome, worse than any amount of analyst toil. Second, every accuracy claim arrives attached to the sample it was computed on, or it does not exist. That took an afternoon and saved the following six weeks.

They also drew a boundary against a familiar neighbour. Their own world applies machine learning to pipeline reliability: event correlation, noise suppression, failure prediction. Security data refuses to behave that way because the source is adversarial — whatever pattern you publish, someone scripts around it by morning, and one missed intrusion can cost more than a year’s tooling budget. They named the difference in a single kickoff-deck line: reliability tooling drifts with the weather, security tooling drifts with an opponent who reads your meeting notes. Budget holders nodded, and nobody asked for “percent automated” again.

The Shadow Fortnight, Blind Labels and All

Behaviour analytics was no newcomer either; analysts had been buying UEBA products as a category since roughly 2015, so the team’s skepticism was earned rather than reflexive. They configured the triage assistant to observe only, exported eighteen months of analyst dispositions, and had one person relabel a sample without sight of the machine’s verdicts. Labels first, scores second, comparison third.

Relabelling was duller than it sounds and more revealing. Eighteen months of dispositions turned out to be inconsistent — the same phishing family closed as benign in March and malicious in July, mostly because two analysts had quietly disagreed about a policy change nobody wrote down. Before the assistant could be judged, the humans had to agree with themselves. Half a week of reconciliation later, ground truth was finally firm enough to grade against, and the exercise paid an unadvertised dividend: the team’s own runbooks improved.

The first honest result landed within a day: pooled accuracy was flattering and nearly useless. Split by category, the tool’s cloud-IAM decisions missed 42.3% of genuine cases while its phishing decisions missed 9.1% — the blended average had been hiding the worst category inside the best one. From that afternoon onward, the team rejected any metric not computed per category.

Where AIOps for Security Earns and Loses Trust

With per-category numbers in hand, gate discipline became concrete. The recommendation tier was straightforward: backed by 200-plus labelled alerts and 30-plus true positives, ranked suggestions were worth an analyst’s glance. Auto-close was another matter. The team’s own blind-label exercise surfaced two misses among 71 true positives — a 2.8% raw rate — yet the statistical upper bound sat near 8.6%, comfortably past the 2% ceiling, because small samples make raw rates liars. Zero errors across n cases pins the true rate only near 3/n; demonstrating a 2% bound with one observed miss takes 236 samples, and with two misses, 313. Their write-up recorded the trio that surprised them most: close decisions ran 99.8% precise, true-positive verdicts 86%, and the share of analyst workload genuinely removable 93.5%. All real. None of it sufficient for autonomy.

Random spot-checks fared no better in theory. Dip into a twentieth of the quarter’s 1,845-odd closes, find nothing wrong, and the mathematics still permits a true miss rate near 38% — comfort journeys prove little, seeded cases prove plenty. So the team planted canaries instead: fifty per quarter, where three closed by the machine means the privilege is pulled, alongside a blind holdout and a reopen path for every automated verdict.

Benchmarks, Read Without Blushing

Public evidence got the same treatment as sales slides. In ITBench, the ICML 2025 benchmark, agents resolved 25.2% of compliance-and-security scenarios. Meta and CrowdStrike’s CyberSOCEval suite told a similar story: agent scores of roughly 23% to 34% on malware analysis work, 43% to 53% on threat-intel reasoning — well above the 0.63% and 1.7% that pure guessing manages, yet nowhere near autonomy. Gartner even has a name for vendors who blur this gap: “AI washing”, dressing deterministic rule engines up as learners. One question joined every procurement call from then on: show us the confusion matrix, per category, on traffic shaped like ours.

Residency and Upload Traps the Brochure Skips

Two practical discoveries never appeared in any deck. First, residency toggles: Microsoft Security Copilot’s customer-data sharing defaults to ON — turning it off requires the M365 E5 or E7 tier — while Gemini in Security Operations processes prompts on global endpoints, though Google states those prompts are not used for training. Second, uploads: forwarding a suspicious attachment to a sandbox or VirusTotal can propagate that file onward; VirusTotal’s own terms note contents “may also be shared with premium VirusTotal customers”. House rules followed: any agent reading attacker-controlled content gets no outbound channels, and nothing leaves the tenant without a named approver.

The House Rules That Survived the Trial

Post-mortems at Merwick usually die inside a shared drive. This one became a wall poster, five lines long:

1. Measure per category, always; a blended score is a hiding place.
2. A miss outranks a toil-saving. The machine may spare work; it may not create breaches.
3. Every autonomy claim arrives with its sample size, or gets laughed out of the room.
4. No agent that reads attacker-controlled content gets an outbound channel.
5. Privileges are rented, never owned: canary seeds renew the lease each quarter.

None of the five lines is technical. All five are governance, and every one was earned the expensive way — mostly by nearly believing a pooled accuracy score in week one.

AIOps for Security, Six Weeks Later

Six weeks in, the assistant stayed an L1 recommender — ranked queues, drafted summaries, a minute-scale first pass on every alert, humans holding every close decision. Toil in the two measured categories fell by more than nine-tenths, and nobody lost a Friday to the swamp again. The team’s final report leaned on the Ops Atlas page describing AIOps for security for the evaluation maths, because it states plainly what the marketing does not: without base rates, bounds, and per-category evidence, “automated” is only an adjective.

What Carries Back Into Data Work

Curiously, the discipline improved their day jobs too. The alert-accuracy practices their platform already ran mapped one-to-one onto blind labelling, and their pipeline-risk review routines turned out to be canary thinking wearing a different name. Their closing slide borrowed a line from earlier reading on automation agents in business workflows: autonomy is granted by evidence, one action class at a time. For people who keep production data honest for a living, that was the only acceptable ending.