Let me tell you about one analytics engineer’s Thursday. She is made up, and her team is a composite, so please do not go looking for them. She spends her days keeping warehouse models honest, and one afternoon the security group next door wandered over with a problem: their detection rules lived in a web console, nobody remembered who had changed what, and a recent alert storm had been traced to somebody’s late-night edit. “You people review everything before it ships,” they said. “Can you show us how?” What followed is a neat little introduction to DetectionOps, and it turns out data folks already know most of it.

First surprise: the rules were the data model

The engineer’s first move was to ask where the rules lived. The honest answer was “in the tool, mostly.” She nodded the way you nod when someone tells you their transformations live in a spreadsheet. In the language of the Ops Atlas explanation of DetectionOps, the first step is to export everything into Git, run a basic lint in continuous integration, and start treating the repository as the place where truth is kept. The Atlas puts that at the thirty-day mark of a thirty-sixty-ninety plan, and the engineer found it reassuring that the first month asks for nothing fancier than a tidy folder and a syntax check.

She showed the security team how her own project keeps models as plain text files with a pull request for every change. The parallel is almost one-to-one. A detection rule, like a model, has a name, a purpose, inputs it expects and a body of logic. The Atlas even mentions an open specification called Sigma that gives rules a common shape: a title, a description of the log source, and the detection condition itself. Each rule carries a randomly generated identifier and a status such as experimental, test, stable or deprecated. The engineer laughed, because her team’s models carry maturity tags in much the same spirit.

Second surprise: “it ran fine” is not a test

The next question came naturally: how do you know a rule works? The security folks said they ran it and watched. The engineer winced. In her world that is like declaring a pipeline healthy because it did not crash.

She walked them through the habit she trusts, which has a close cousin in the Atlas: test first. Each rule needs at least one example that ought to trigger it and one that ought not to. Once you have those, a pull request can run them automatically. If you have read our piece on practical data validation inside a DataOps workflow, you will recognise the rhythm: define expectations, run them on every change, fail loudly when they break. The sixty-day goal in the Atlas is modest and specific: cover the twenty noisiest rules with tests and start recording how analysts dispose of the alerts, so the information flows back into tuning.

The engineer added one warning from her own scars. If you derive the test from the rule’s output, it passes by construction. Write the example from the behaviour you want to catch, never from what the rule currently does.

Third surprise: staging exists here too

Then came the moment she enjoyed most. “Where do you try a new rule before it pages anyone?” Silence. In data work we use a development schema, a preview build, a slice of yesterday’s partition. For detection, the Atlas describes shadow mode: the rule runs against real traffic for roughly a week, records what it would have flagged, and pages no one. At the end the team looks at the hits, tunes, and only then lets it alert. One practical touch from the entry is that the pull request itself can trigger the staging deployment and the merge can trigger production, so nobody has to remember a separate manual step.

The security team’s reaction was telling. They had been afraid that adding gates would slow them down. The engineer pointed out the Atlas’s rule for genuine emergencies, which she found sensible: during an active campaign a change may skip the shadow week, but never the tests or the peer check, and it gets a second look inside five working days. Speed gets its lane, and discipline stays on.

Fourth surprise: the metrics looked familiar

The engineer loves a dashboard, so she asked what they measured. The Atlas answer is a short list of ratios she could have invented herself. Test completeness is the share of rules that have tests, and in the entry’s example it sits at 84 of 120 rules, or seventy per cent. Rule health is the share not stale or broken, shown as 114 of 120. Change failure rate is how many deployments needed a rollback, with a worked figure of three in fifty.

The one she copied onto a sticky note was precision per rule, true alerts divided by everything the rule raised. In the example, a rule with forty alerts and six true ones scores fifteen per cent, which she translated into data-quality terms: that is a check with an eighty-five per cent noise rate, and nobody would keep it. She also liked the Atlas’s cautionary line on coverage. Tagging rules with a technique code from the ATT&CK catalogue does not prove the technique would be caught; the entry’s illustration has a tag heatmap showing everything covered while emulation caught only 60 per cent of the prioritised cases.

What she told them at the end

Over coffee the engineer summarised it in plain words, and I will do the same, because it is the part you can take back to your own team.

• Put the rules in version control first. Everything else depends on that.
• Give every rule a purpose line and a couple of behaviour-based tests.
• Try new rules quietly before letting them page anyone.
• Count a few ratios monthly and look at the trend, not a single figure.
• Record every exception, such as an exclusion that hides a noisy account, with an owner and an end date, since each exclusion trades a little visibility for fewer false alarms.

She also warned against grand platforms. The entry argues that a small lab can begin with a Git repository, two scripts, a CI runner and a single API, which sounded very like the starter kit she once assembled for her own team. And because dbt-style thinking was already on her mind, she pointed them to our walk-through of building reliable transformations with dbt as a feel for how version-controlled logic behaves day to day.

The Atlas is careful about scope, and so is this story: the security team still owns the alert queue, the hunting, the incident response and the log collection. The engineering practice described here covers only the rules and the evidence around them. The engineer’s contribution was a mindset borrowed from data work, not a takeover.

The most useful thing, she decided afterwards, was reading the entry’s own caveat: the label DetectionOps is not an official standard, and the same ideas travel under the names detection engineering and detection-as-code. If you go searching, use those words too.