No such team exists in real life, yet its troubles will be familiar to anyone who has maintained a content-heavy website. Call them the platform group at an imaginary publisher running several CMS-driven sites. Their editors, developers and a two-person security desk shared one headache: the rules that were supposed to warn them about suspicious activity on the servers behind those sites had grown without any discipline. The group eventually adopted what the Ops Atlas calls DetectionOps, and the sequence of events below shows what changed and why.

Chapter one: the spreadsheet years

At the start, the security desk kept its threat notes in a shared spreadsheet and its alert rules inside the monitoring console. A row in the sheet might say “watch for new scheduled tasks on the publishing servers,” but the console held no pointer back to that row. Months later nobody could say which rule answered which worry, or who had last touched it.

The Atlas write-up of DetectionOps describes a very similar situation at a European public institution before it adopted a structured approach: threat information lived in spreadsheets and the links to detection rules were lost. The Atlas adds a caveat that deserves repeating here, namely that this account comes from the tool’s own side and has not been independently evaluated. So treat it as an illustration of a pattern, not a proof of results.

Chapter two: a release that nobody could explain

One Friday the desk silenced a noisy rule by editing it directly in the console. The following Tuesday an automated sync restored the previous version, and the alert flood came back during the busiest editorial hour. A developer, used to code reviews and rollbacks on the website itself, asked the obvious question: why do our website changes have history, but our alarms do not?

That question was the turning point. The group decided to hold detection logic to the same standard as the site’s own templates and plugins. If a content type change needs a pull request, so does a rule change.

Chapter three: moving the rules into a repository

The first month was unglamorous. The team exported every rule into text files, committed them, and added an automatic format check that ran whenever a file changed. For rule syntax they considered the open Sigma format, which describes each rule with a title, the log source and a condition. Because they ran a single monitoring platform with platform-neutral requirements, they followed the Atlas’s judgement for that case and kept the native query language as code, skipping the conversion layer.

Pinned versions came next. A teammate noticed that a major release of the Python library used by rule converters had just appeared while two companion tools still required the previous line. Pinning every dependency meant the build stayed predictable while they planned an upgrade deliberately.

Chapter four: tests that were not written by the rule’s author alone

The group adopted a minimum of two tests per rule: one event that must trigger it and one lookalike that must not. A pairing habit emerged, where a developer from the web team wrote the harmless examples because they knew what normal publishing activity looked like. Their input exposed a rule that treated every nightly cache purge as suspicious.

They also began recording accepted blind spots as tests, each with an owner and a review date. When the desk filtered out a backup service that generated noise, the filter and its documented gap were committed together. This mirrors how the same team had learned to plan a website thoroughly before development begins: write down what you are deliberately leaving out.

Chapter five: shadow mode and the first honest numbers

New rules now ran in shadow mode for about a week, producing a record of would-be alerts without paging anyone. After one such week, the desk compared its sample of results with expectations. One rule produced forty alerts in the month, of which six were genuine, giving fifteen percent precision, with each real finding costing about forty minutes. The figures echo the Atlas’s worked example, which the team used as a template for its own dashboard.

When management asked how well the sites were covered, the desk resisted quoting the tag heat map. Instead they picked five priority behaviours, emulated them against a test environment, and reported which were detected. The Atlas notes that framework tags alone are not proof of coverage, and the exercise showed why: two of the five went unnoticed despite having tagged rules.

Chapter five and a half: the metrics meeting

Once the numbers existed, the group held a short monthly review. The agenda never changed. First, the noisiest five rules, with the cost in analyst minutes next to each. Second, test completeness across the library, which in the Atlas example stood at 84 of 120 rules, or seventy percent, before the maturity target of ninety. Third, any change that had gone wrong, since three failures in fifty changes, six percent, is the sort of figure worth watching for drift.

The meeting produced decisions rather than slides. A rule that cost forty minutes per genuine finding was either narrowed or retired. A rule with no tests got tests before anything else was touched. A filter nearing its expiry date was either renewed with fresh evidence or removed. Within a quarter the editorial leads stopped asking whether the desk was busy and started asking which rule was next in the queue, a much healthier question.

Chapter six: what the group would tell others

• Do not try to reach the top maturity tier at once. The Atlas parallels Elastic’s published model in warning that rushing causes burnout, and suggests reaching the validated tier for the most important threats first.
• Keep changes of every urgency inside the same path. Emergencies may skip the waiting period, but never the tests or review.
• Retire rules openly, marking them deprecated, instead of leaving dead logic around.
• Share the metrics with editors and developers. Fewer false pages is a benefit they feel immediately.

Epilogue: the habit that outlasted the project

A year on, the imaginary group no longer thought of this as a security initiative. It was part of how the platform team worked, as normal as peer review on theme changes. Their editorial colleagues noticed that incidents got shorter and quieter. The security desk stopped spending Friday afternoons arguing over which rule was right.

If you manage a content platform and want a parallel on structure, our article about the pages every business website needs shows the same mindset applied to site architecture: decide what must exist, document it, and review it regularly. Detection logic deserves precisely the same care.