Detection engineering is a software problem

Ask a security operations centre how good its detection coverage is and the answer usually arrives as a number of rules. Two thousand, four thousand, twelve thousand. The number goes up over time, which feels like progress, and it can be reported to people who do not have the context to ask what it means.

It means very little. A rule that references a field the pipeline stopped populating eight months ago still counts. So does a rule that fires four hundred times a day and is closed unread, a rule that duplicates three others, and a rule written for a product that has since been decommissioned. The count measures how much content has been written. Coverage is a claim about what would be caught, and nothing about a rule count supports it.

What a detection actually is

A detection has three parts, and only one of them is the query. There is a hypothesis about adversary behaviour — what an attacker would have to do, and what observable trace that leaves. There is a data contract — the specific fields, from the specific sources, that must be present and correctly populated for the trace to be visible at all. And there is the logic that expresses the first in terms of the second.

Written down that way, the failure modes become obvious. A detection can be wrong because the hypothesis is wrong, because the data contract is unmet, or because the logic does not say what its author thought it said. These are different problems with different owners and different fixes, and a rule stored as a row in a console flattens them into one undifferentiated thing called content.

Detections have dependencies

Software engineering learned to track dependencies because a change somewhere else can break your code without touching it. Detection content has exactly this property and almost never tracks it. A rule depends on a log source continuing to arrive, on a parser continuing to extract the same fields, on a vendor not renaming an event type in a minor release, and on the enrichment that turns an IP address into an asset continuing to resolve.

When one of those changes, the rule does not fail loudly. It stops matching, which looks identical to an absence of attacks. This is the single most consequential difference between detection logic and ordinary application code: the default failure mode is silence, and silence is indistinguishable from success.

The practical response is to make the data contract explicit and monitored. If a detection declares the sources and fields it requires, then a pipeline change that breaks that contract can raise an alert about the detection rather than waiting for an incident to reveal it. This is not sophisticated work. It requires deciding that the dependency is worth writing down.

Tests, and what they can honestly establish

A detection test replays a known event sequence and asserts that the rule fires. A good suite also asserts that it does not fire on a defined set of benign sequences, which is the half that tends to be skipped and the half that governs whether analysts trust the output.

It is worth being precise about what such a test proves. It proves that the logic matches the events it was given. It does not prove that an adversary's real behaviour resembles those events, and a suite of tests written by the same person who wrote the rule tends to encode the same assumptions twice. Tests catch regression reliably and catch mistaken hypotheses hardly at all — which is still a large improvement over having neither.

Retirement is a feature

Content libraries grow monotonically because deleting a rule feels like reducing coverage, and because nobody can reconstruct what a five-year-old rule was defending against. The hypothesis was never recorded, so the only safe action is to leave it running.

The cost is paid twice. Once in triage, where dead content generates volume that displaces attention. And once in assessment, because a library nobody is willing to prune cannot be described accurately — every review starts by rediscovering what is in it.

A detection whose hypothesis is unrecorded cannot be evaluated, only inherited.

Writing the hypothesis down at authoring time costs a paragraph. It is what makes retirement a decision rather than a risk, and it is the input a coverage discussion actually needs.

What changes in the SOC

Three things follow from treating detections as software, and they are organisational before they are technical.

1. Detection content moves into version control and is promoted through environments, which means the question who changed this and why has an answer.
2. The data platform and the detection library stop being separate concerns owned by separate teams, because a data contract spans both.
3. Coverage becomes a statement about tested hypotheses against a named threat model, which is a smaller number than the rule count and a defensible one.

The third point is the one that meets resistance, because the honest number is always lower than the number currently being reported. It is also the only version of the number that survives contact with an incident review.