Outage confirmation
One region says down.Sentinel asks the world.
Every incident Sentinel opens has been confirmed by at least half of its regions, on at least two continents. This is the rule, what happens in the seconds after a first failure, and why it works this way.
From first failure to alert
one outage
- failFrankfurt's scheduled check fails
- verifyOther regions are asked to check right now
- agreeEnough of them fail too, on more than one continent
- waitThe confirmation period passes and the regions look again
- alertStill down. The incident opens and alerts go out
Why one failure isn't enough
A failed check is a question.
A check from one place can fail for reasons that have nothing to do with your site. A route flaps, a data center's upstream drops packets, a CDN edge near one city misbehaves, or a firewall rate-limits one range of addresses.
Alert on every one of those and people learn to ignore the alerts, which is worse than having none. So Sentinel never opens an incident on one region's word. It asks the others first.
Alert decision
- Regions checked
- 15
- Failing
- 1 (Frankfurt)
- Needed for an outage
- 8, on two continents
- Verdict
- Regional, no alert
When the others can still reach the site, Frankfurt's failure stays on the monitor for you to see. Nobody gets paged.
The rule
Three conditions, all true together.
An incident opens only when every one of these is true. It closes when they stop being true. HTTP, ping and port monitors all use the same test.
Half
At least half the regions
Half the regions checking the monitor, rounded up. On the full network that's 8 of 15, and a monitor on five regions needs three.
3+
Three or more
At least three failing regions whenever the monitor uses three or more. A monitor on one or two regions needs all of them.
2 continents
More than one continent
The failing regions have to span two continents whenever the monitor's regions do. A rule that blocks all of Europe is regional, not an outage.
The first few seconds
Verification doesn't wait its turn.
Scheduled checks are spread out across each interval, so your server never takes fifteen requests in the same instant. Verification isn't spread out. The moment one region's check fails, Sentinel asks enough other regions to check immediately, without waiting for their own schedule.
The answer comes back in seconds, whether the monitor runs every 15 seconds or every five minutes. The check interval decides how soon a failure is first seen. It never decides how long confirming it takes.
Confirmation and recovery
Then it waits, on purpose.
- Failure
- Verification
- Confirmation
- Incident
- Recovery
- Resolved
30s default
Confirmation period
Once the rule is met, Sentinel holds the alert for the monitor's confirmation period. At the end, the regions check again. If the site is back, no incident opens and the monitor records a blip, so you can still see it happened.
1 minute default
Recovery period
An incident closes when failures drop below the rule and stay there for the recovery period. Then every region checks once more before it closes. A site that flaps for ten minutes gets one incident and one pair of alerts, not ten.
Both periods are set per monitor, up to five minutes for confirmation and thirty for recovery, in the dashboard, the API, the CLI and Terraform. A monitor on a single region alerts on its first failed check unless you give it a confirmation period. There's no one for it to agree with, and that's what single-region monitors are for.
A region that can't reach your site doesn't page anyone, but its result stays on the monitor beside every other region's, with the timing and the failure it saw. You can read a regional problem without being woken up by it.
The disagreement is worth reading. Eight regions failing while seven succeed is a different story from fifteen failing together. Failures confined to Europe point toward a regional network, CDN or routing problem. DNS failing everywhere looks nothing like DNS resolving and the connection failing after it. One region timing out while fourteen connect normally is evidence even when it never becomes an incident.
What is outage confirmation in Sentinel?
Outage confirmation is how Sentinel by Rootstuff decides a failure is a real outage before it alerts anyone. When one region's check fails, other regions check the same target immediately. An incident opens only if at least half of the monitor's regions fail, three or more of them, across at least two continents, and the failure lasts through the confirmation period. Failures that don't meet that bar stay visible on the monitor without paging anyone.
Does every Sentinel plan include outage confirmation?
Yes. Every plan, the free plan included, checks from all 15 regions and uses the same outage confirmation for HTTP, ping and port monitors. Paid plans check more often, down to every 15 seconds on Business, but the rule and the immediate verification are the same everywhere.
How many regions have to fail before Sentinel alerts?
At least half of the regions checking the monitor, three or more, spanning two continents when the monitor's regions do. With all 15 regions that's 8.
How long from the first failed check to an alert?
The verification checks run straight away, so the rule is usually settled within seconds of the first failure. The confirmation period then holds the alert, 30 seconds by default, and the regions look once more before the incident opens.
Does a five-minute monitor wait five minutes to confirm?
No. The check interval decides how soon a failure is first seen. Confirmation starts the moment it is, on every plan, with checks from other regions that don't wait for their own turn.
What if my site blocks a country or a region?
If the failing regions are fewer than half, or all on one continent, Sentinel treats it as regional. Those regions' results show the failure on the monitor, and nobody is paged.
Can I make it stricter or looser?
The rule is the same for every monitor, but you choose which regions a monitor uses and how long its confirmation and recovery periods are. A monitor on a single region alerts on its own first failure, which is how you get alerts for one region specifically.
Do ping and port monitors work the same way?
Yes. HTTP, ping and port monitors share the rule and the immediate verification. Heartbeat and cron monitors are push-based, so they use a grace period instead.
Get paged for outages, not blips.
The full rule, every region and immediate verification on every monitor, the free plan included.
No card for the free plan · 14-day trial on paid plans · Cancel anytime