A suppression nobody remembers
During an investigation, an analyst notices that alerts from a group of a client's servers have been suppressed for a particular behaviour. The suppression was added more than a year ago. Why? The ticket history mentions a noisy backup job. The client's IT lead at the time agreed it on a call. They have since left. The backup job was replaced months ago. The suppression is still there.
This is not unusual. Tuning is a constant part of running a SOC. Noisy sources are tuned, exceptions are added for known software, thresholds are adjusted. Most of it is done directly in the console by the analyst who dealt with it, with a note in a ticket if you are lucky.
Over time, each client's environment carries a layer of exceptions that nobody could fully list or explain.
The client, too, has a stake. They are relying on your monitoring. Changes that reduce what is alerted on are decisions they should be part of, and should be able to see later.
Why tuning records go missing
- Changes are made directly in the SIEM or EDR console, which is the quickest place to do it.
- Client approval is given verbally or in an email, not recorded against the change.
- The reason for a change is known to the analyst who made it and nobody else.
- Suppressions have no expiry or review date.
- Each tool records changes differently, if at all.
What undocumented tuning costs
| Gap | Consequence |
|---|---|
| No record of approval | Unclear who agreed to reduce alerting |
| No reason recorded | Nobody knows if the suppression is still needed |
| No review date | Old exceptions linger after circumstances change |
| Changes spread across tools | No single view per client |
| Analyst leaves | Knowledge of why things were tuned goes with them |
If something is missed and the question is asked why a detection did not fire, a clear record of what was tuned, why and with whose agreement is exactly what you want to have.
The tuning register we build
- Tuning requests are raised through a short form in your PSA or a small tool: client, rule or source, proposed change, reason, and whether it is temporary or permanent.
- Changes that reduce alerting are sent to the client's nominated approver for sign-off, with the reason in plain language. Approval is recorded with name and date.
- Each approved change gets a review date according to your policy. Temporary changes get an expiry.
- The register links each record to the rule or exclusion in your SIEM or EDR platform, and a regular check compares the register with what is actually configured, flagging changes made in the console with no record.
- Reviews are prompted to the analyst team and, where needed, back to the client, with the original reason shown.
- Each client's current exceptions can be listed at any time, and included in their service review pack.
Your policy decides which changes need client approval and how often they are reviewed. We build to it.
What the SOC has afterwards
Every exception in a client's environment has a reason, an approver and a review date. Analysts can see why something was tuned before they change it again. Clients see their exceptions in their service review and confirm or remove them. Old suppressions are reviewed rather than forgotten.
And the regular comparison catches the change made in a hurry in the console, so it can be recorded or reversed.
Here is a normal week. An analyst raises a request to tune alerts from a client's new software deployment tool. The client's approver signs off the next morning and the change is applied with a three-month review date. The weekly comparison flags an exclusion added in the console on Friday night with no record; the analyst who made it adds the reason and it goes to the client for approval. Nothing about the week was unusual, and everything about it is on the record.
Is your tuning documented?
- Suppressions are added directly in the console with no formal record.
- Client approval for tuning is verbal or in email.
- You have found old suppressions nobody could explain.
- There are no review dates on exceptions.
- You cannot list a client's current exceptions quickly.