Noise control
How detection thresholds, deduplication, throttling, flap suppression, quiet hours, quorum, dependencies, and correlation keep alerting quiet.
On this page
- Why this page exists
- Before an alert opens: detection sensitivity
- Partial outages are downgraded, not silenced
- After an alert opens: deduplication and repeats
- Flap suppression
- Recovery period (auto-resolve hold)
- Slow-response warning (opt-in)
- Quiet hours
- Alerts from integrations
- Grouping and suppression across monitors
- Mass-outage digest
Why this page exists
A good on-call tool is judged as much by the pages it doesn't send as the ones it does. HowlOps has several independent mechanisms that reduce noise, and they act at different stages of the pipeline. This page collects all of them so you can see what is dampening (or could dampen) an alert. For the end-to-end flow, see Alert & incident flow.
Before an alert opens: detection sensitivity
These decide whether a failing check becomes an alert at all.
| Control | What it does | Where |
|---|---|---|
| Confirmation threshold | Requires N consecutive failed checks before an alert opens (default 2). | Per monitor |
| Retry on fail | The prober immediately re-checks, so one confirmed failure suffices. | Per monitor |
| Region quorum | Only open an alert when at least X% of regions are down. Below that, it's a partial outage and stays quiet. | Per monitor (multi-region) |
The confirmation checks (and extra regions agreeing) only confirm the outage is real; they do not push its start later. Once an alert opens, its start time is backdated to the first failing check of the outage, so the incident timeline and the response-time chart mark the problem from the moment it actually began, not from the moment it was confirmed.
Partial outages are downgraded, not silenced
When a monitor runs from several regions in the default (deduplicated) mode and only some
regions fail, HowlOps treats it as a partial outage: the alert opens at warning
severity instead of critical, so it flows through your normal channels as a lower-priority
signal rather than a full page. If a partial outage should page like a full outage for a
given monitor, turn on Treat partial outages as critical (partial_outage_critical) on
that monitor. See Monitors → Degraded state.
After an alert opens: deduplication and repeats
| Control | What it does | Default |
|---|---|---|
| Alert deduplication | While an alert is open, further failed checks for the same monitor do not open new alerts. You get one page for the outage and one for recovery. | Always on |
| Per-channel throttle | The same alert is not re-sent to the same channel within a window. | 5 minutes per channel + event |
| Renotify interval | Opt-in. Re-send an unacknowledged alert every N minutes as a reminder during a long outage. | Off (per monitor) |
Flap suppression
A monitor that bounces up and down repeatedly ("flapping") would otherwise storm your channels. HowlOps detects it and dampens the noise without hiding a real outage:
- If a monitor opens 5 or more alerts within a 10-minute window, it is marked flapping.
- While flapping, the broadcast to channels and the status-page updates are muted and a single "flapping" notice is posted instead. There is no fixed end time: muting is re-evaluated on every check, so it lifts as soon as the monitor's opens over the trailing 10 minutes drop back below 5, and it keeps going for as long as the monitor keeps flipping. A 30-minute cooldown only limits how often the "flapping" notice itself can repeat, not how long muting lasts.
- Escalation is deliberately not suppressed. A genuine sustained outage that begins during a flap window must still page on-call, so the escalation engine keeps running.
The same damping applies to alerts that arrive from your own monitoring. If one alert from Alertmanager, Prometheus, Grafana or a webhook fires and clears five or more times within ten minutes, HowlOps sends a single notice instead of a message for each flip. It is matched on the alert's fingerprint, so one noisy alert never silences a different one, and every occurrence is still recorded and visible in the alert's history. Escalation is not suppressed here either.
An alert that flaps is usually best settled at the source — a for: clause on the rule that
sends it will stop it firing until the condition has held for a while. The notice we send says
so, because that fix is worth more than our damping.
Recovery period (auto-resolve hold)
By default an alert resolves as soon as the monitor is back up. For a service that recovers for a few seconds and then fails again, that can split one bad episode into several separate incidents.
Set a recovery period (recovery_period_s) on a monitor to require it to stay continuously up
for that many seconds before the incident auto-resolves. A brief recovery no longer closes the
incident, so a flapping outage stays a single incident instead of many. Blank or 0 keeps the
default (resolve immediately). It's on the monitor's edit page under Recovery period (seconds).
Slow-response warning (opt-in)
Separate from up/down, a monitor can be degraded: reachable but slow. If you set a slow-response threshold on a monitor, a check that comes back up but over that threshold marks the monitor degraded and sends a low-severity warning notification. It does not open an incident and does not page or escalate, and it honours quiet hours. To avoid noise, at most one warning is sent per cooldown window (your renotify interval, or 30 minutes) while the monitor stays slow. It's fully opt-in: with no threshold set, a monitor is never marked degraded on speed alone. You can also turn the degraded event off per channel in your notification preferences.
Quiet hours
Quiet hours hold non-critical notifications during a window you define (critical always pages). Enforcement differs by channel. This is intentional because a real page should wake you, but worth knowing:
| Channel | During quiet hours (non-critical) |
|---|---|
| Chat (Slack / Discord / Telegram) | Dropped |
| SMS | Dropped |
| Held for the daily digest (when the digest is enabled) | |
| Mobile push | Dropped (suppressed during any active quiet-hours window, independent of the email digest) |
Critical-severity alerts bypass quiet hours entirely. Escalation pages are treated as critical, so quiet hours effectively only mute warning/info broadcasts. There are two independent windows: a workspace-wide window (Settings → Notifications → Quiet hours) and an optional per-user window.
Windows are set to the minute, not to the hour. A window of 22:30 to 07:15 is exactly that. The start is included and the end is not, so a window ending at 07:15 is over at 07:15. A window whose start and end are the same is treated as no window at all rather than as a silent day, because that is what a half-filled form usually means.
Alerts from integrations
Alerts that arrive from your own monitoring (Alertmanager, Prometheus, Grafana, a webhook) follow the workspace window by default, the same as everything else. Most of them arrive as warning rather than critical, so in practice a non-critical alert from your Prometheus that fires at 02:00 is held until the window ends.
If that is not what you want, turn off Apply quiet hours to alerts from integrations in Settings → Notifications → Quiet hours. Those alerts will then page through the window, the same way a monitor does when you leave its own quiet-hours setting off. Critical alerts are unaffected either way: they always get through.
The workspace-wide window mutes alerting for everyone, so changing it requires permission to edit workspace settings. Owners and Admins have it by default; a custom role needs the settings edit permission. Your own per-user window is always yours to change and needs no extra permission.
Both windows, and the daily email digest send-time, are evaluated in a timezone. By default that's the workspace timezone (Settings → Workspace) for everyone. If you personally sit in a different zone than the rest of your team, set your own timezone under Settings → Profile → Time zone: once set, it overrides the workspace timezone for your quiet-hours windows, your digest timing, and the times printed in your on-call shift reminders. Teammates without a personal timezone are unaffected. Leave it on "Use workspace default" to keep following the team's zone.
Grouping and suppression across monitors
| Mechanism | Effect | Reduces pages? |
|---|---|---|
| Correlation | Alerts that fire close together are grouped into one view with an "N monitors affected" banner. | No by default: a display grouping only, and each alert still fires. |
| Mass-outage digest | When a correlation group grows past a threshold you choose, the group is announced once — how many services are affected, which ones, and a link to the list — and the individual alerts for the rest of the outage are held. | Yes, when you turn it on: the first alert of the outage still pages on-call, the rest are held. |
| Monitor dependencies | While a parent (upstream) monitor is down, alerts for the monitors that depend on it are suppressed. | Yes: child pages are withheld; the parent's outage already paged you. |
| Maintenance windows | During a scheduled window, alerts for the selected monitors are suppressed (checks still run). | Yes: for the window's duration. |
To cut down the number of pages during a shared-cause outage you have two tools, and they solve different halves of the problem. Use monitor dependencies (see Monitors) when you know in advance which service depends on which. Use the mass-outage digest when you do not — it reacts to scale rather than to a declared relationship.
Mass-outage digest
Correlation has always been able to tell that many of your services failed together. The digest is what happens next: past a threshold you choose, HowlOps sends one message for the group instead of one per service, and repeats it every 30 minutes while the outage lasts so you can tell "still burning" from "fixed and nobody told me". When the last incident closes, you get one all-clear naming how many services were affected.
Three things it deliberately does not do:
- It never stops paging. The first alert of the outage goes through your escalation policy exactly as it would otherwise, so somebody is always woken. The digest tells that person the scale; it does not replace their page.
- It never loses an incident. Every affected service still opens its own incident, appears in your incident list, keeps its timeline and its history. What is held is the message, never the record.
- It never sends an all-clear for a page you did not get. A service whose alert was held does not send you its individual recovery either — the group's all-clear covers it.
It covers inbound integrations too. Alerts from Alertmanager, Sentry and the rest are grouped by their source and their alert name, so one rule firing against many targets — the usual shape of a mass outage — becomes one summary. Two different rules stay two groups on purpose: merging unrelated failures would silence the alerts about the second one, and an outage you hear about too loudly is better than one you do not hear about at all.
Turn it on per workspace in Settings → Alerts → Links & dedup, where you also set how many simultaneous incidents count as a mass outage. It is off until you switch it on.
Switching it off stops the digest at once, but alerts that are already being held come back as their holds expire, which takes up to 35 minutes. The same applies when an outage shrinks: once fewer services are affected than your threshold, it stops being treated as a mass outage and the remaining alerts return to normal delivery as their holds lapse. There is no state in which an alert stays held indefinitely.
Was this page helpful?