Alert & incident flow

The big picture

Every signal HowlOps reacts to follows the same path. The source differs (a monitor going down, an external alert arriving, a heartbeat going quiet), but from there the flow is identical:

The rest of this page explains each step, and every option you can configure to change how it behaves.

Alerts and incidents are two levels

HowlOps separates a raw alert from a confirmed incident:

An alert becomes an incident in one of two ways:

1. Automatically, when it matches a promotion rule. Out of the box there is one rule: any alert with critical severity is promoted. You can add your own rules that match on severity, source (monitor, heartbeat, integration), or labels.
2. Manually, with the Promote to incident button on any alert.

Step 1: where alerts come from

Monitor goes down

A single failed check does nothing on its own. A detector runs on each check (as often as every 15 seconds on Uptime L, every 120 seconds on Free) and opens an alert only when the last N checks were all down. You control N:

For monitors that run from several regions you also get:

External alert arrives

Alerts can come from Prometheus Alertmanager, Grafana, Datadog, CloudWatch, or a generic webhook. HowlOps normalises the payload to a name, a state, a severity, and a fingerprint (a stable key the source provides). Deduplication is keyed on that fingerprint:

• A new fingerprint opens a new alert.
• A repeat of the same fingerprint updates it and bumps a "fired N times" counter. It does not open a duplicate or re-notify.

External alerts have no monitor of their own, so there is no screenshot, prober log, or recheck for them.

Heartbeat is missed

A heartbeat expects a ping on a schedule. It is marked down when:

• the last ping is older than interval + grace, or
• it was never pinged and is older than twice its interval, or
• it missed its cron-scheduled time plus grace.

The alert is titled with the heartbeat's name and carries its configured priority (default P1). It is idempotent: while one alert is open, missing again does not open another.

One alert per incident (deduplication)

Whatever the source, you are notified once when something breaks and once when it recovers — never on every failed check. Subsequent failures for the same open alert are deduplicated:

14:00  check fails      → alert sent (incident opens)
14:10  check fails again → no alert (same incident)
14:20  check fails again → no alert
14:30  check succeeds    → recovery alert sent (incident resolves)

On a paid On-call plan you can add repeat alerts (a per-monitor renotify interval, see Step 4): while an alert stays open and unacknowledged, HowlOps re-sends it every N minutes as a reminder during long outages.

Step 2: screenshots (HTTP monitors)

If a monitor is HTTP and you enabled Capture screenshots on failure (off by default, paid Uptime plans), HowlOps takes a screenshot of the page a few seconds after the alert opens.

Step 3: how the alert is delivered

The first decision is whether an escalation policy governs the alert. HowlOps can pick one for you automatically:

Configure a routing rule

A routing rule is a condition + target pair. When a new alert is created, HowlOps evaluates your rules in ascending priority order (1 → 2 → 3 …). The first matching rule wins and pins its escalation policy; the rest are skipped. If no rule matches, the alert falls back to the monitor's own policy, then the workspace-level default escalation policy.

To create one, go to Alerts → Routing Rules → New Rule, then set a condition, a match value, the escalation policy to apply, and a priority (lower number = evaluated first).

New alert created
    │
    ▼
Rule 1 (priority 1): monitor_tag = "payments"   → match? YES → apply "Critical on-call" → done
    │ NO
    ▼
Rule 2 (priority 2): monitor_type = "heartbeat" → match? YES → apply "Heartbeat on-call" → done
    │ NO
    ▼
No more rules → fallback: workspace default escalation policy

A policy governs the alert

The immediate broadcast is skipped (so you are not notified twice). The escalation engine takes over and pages on-call step by step over time (see Step 4).

No policy governs the alert

The alert is broadcast to every enabled channel plus mobile push. Each channel still passes through a set of filters and can be skipped:

Step 4: escalation steps and your personal chain

When a policy governs the alert, two layers of timing apply, and both are yours to configure.

Escalation steps run in order. Each step has its own delay and its own target:

Step 1  delay 0 min    page the on-call engineer
Step 2  delay 15 min   also page the secondary
Step 3  delay 30 min   notify the team lead
(repeat the policy up to N times, or stop)

Your personal chain decides how each person is reached. Each channel has a priority and a fallback delay, so you can spread channels over time:

priority 1   push    immediately
priority 2   email   after 300s   (5 minutes)
priority 3   sms     after 600s   (10 minutes)
priority 4   voice   after 900s   (15 minutes)

The chain advances only while the alert is still unacknowledged and not silenced. If you acknowledge at minute 3, the SMS and voice steps never fire.

You can also set a per-monitor renotify interval to re-send an alert every N seconds while it stays unacknowledged.

Step 5: routing labels

Routing rules match on labels. Where those labels come from depends on the source:

• Monitors and heartbeats: HowlOps sets the labels from your configuration (the monitor type, its tags, the severity).
• External integrations: the labels come from the incoming payload. HowlOps reads the labels and tags the source sent and routes on those, so the quality of your routing depends on the source sending good labels.

Step 6: recovery and resolve

When the service recovers, the alert resolves automatically:

• A monitor that returns healthy auto-resolves (multi-region waits for every region).
• An external source that sends a resolved event closes the matching alert by fingerprint.
• A heartbeat that pings again recovers and closes its alert.

The all-clear travels the same path the alert did:

• If a policy paged on-call, only the people who were actually paged get the all-clear.
• If it was a broadcast, the all-clear goes to the channels that received the original alert.

Acknowledge, silence, unacknowledge

What you can configure (summary)

Related

• Incidents & severities
• Notifications
• On-call & escalations
• Escalation policies