Incidents and uptime
Incidents shows the outages and alerts Polo watches, what it found and what it changed. An incident opens when an uptime monitor fails or an alert arrives. Polo applies a fix only after someone approves it, or when a standing approval covers that exact change.
The tabs
Section titled “The tabs”| Tab | What it shows |
|---|---|
| Overview | Active incidents, uptime at a glance, recent incidents and alerts received |
| Incidents | What is failing now, and how each past incident ended |
| Uptime | Your monitors. Add monitor is here. |
| Alert sources | Where incidents come from, and the alerts each source delivered |
Uptime monitors
Section titled “Uptime monitors”A monitor checks a URL on a schedule. After enough failed checks in a row it opens an incident, and after enough good checks it closes it.
Add a monitor
Section titled “Add a monitor”- Open Incidents → Uptime and press Add monitor.
- What to check. Enter a Name and the URL to check. Choose where it Runs from: Polo, over the public internet, or one of your runners. Set Check every (seconds), Failed checks before an incident and Good checks before it closes. Optionally list the Healthy HTTP statuses; leave it empty for any 2xx.
- If the URL is protected, choose how the check signs in: a username and password (basic auth) or a request header such as a bearer token. Polo encrypts it, sends it only to that URL’s host, and never shows it again.
- When it fails. Choose Only open an incident (no agent, no AI cost) or Open an incident and have an agent investigate. For an investigation, choose the agent, the environment and service, the read-only tools it may use, and optionally the fixes Polo may propose.
- Who to tell. Add channels: Slack, Google Chat, Email or Webhook. Polo posts when the incident opens and again when the URL recovers. You can reword both messages, preview them and press Send test message.
- Review and add, then press Add monitor.
Monitor states
Section titled “Monitor states”| State | Meaning |
|---|---|
| Healthy | The latest checks passed |
| Failing | Checks are failing; an incident is open |
| Paused | Polo does not check the URL until you resume it |
| No recent checks | No check has run lately. Check that background jobs are running. |
Each monitor row shows its last check and its uptime over the last 24 hours. Row actions are Edit, Update tools, Pause or Resume, Archive and Investigate now.
- Update tools gives the monitor’s investigations every read-only tool its agent has now.
- Changing the URL closes the open incident and restarts the failure count, the uptime and the check history.
- Archive deletes the monitor’s check history. Its past investigations stay available.
A monitor’s own page
Section titled “A monitor’s own page”Open a monitor to see its numbers (checks passed over the last day and week, and response times), a chart of the last 7 days by hour, the check log with when each check ran, what answered and how fast, every incident with how long it lasted and how it ended, and how the monitor is set up.
Alert sources
Section titled “Alert sources”Alerts reach Polo through Intake.
- Grafana. Add a Webhook integration to a Grafana contact point, with the URL and signing secret Polo shows when you create the source, and list the alert rules you allow. A firing alert opens an incident and an investigation; a resolved notification is recorded on it. Follow Send a Grafana alert to Polo.
- Slack alert bots. A custom Slack app source can list alert channels and the alert bots whose messages Polo should investigate. Slack does not report recovery, so the investigation checks whether the service recovered.
An alert source reads Registered · no delivery yet until its first signed delivery arrives. Polo does not correlate alerts across rules or with uptime incidents, and each Slack alert message opens its own task.
Read an incident
Section titled “Read an incident”Each incident is told as a short story: opened, Polo investigated, a fix proposed, approved and run, recovered, and whether the fix is why.
| What you see | Meaning |
|---|---|
| Waiting for Polo to start | The investigation task is queued |
| Polo couldn’t start an investigation | See the reason; check the alert source or the agent |
| No agent investigates this monitor | The monitor only checks and records the outage |
| No fix proposed yet | Polo is investigating, or reports findings only |
| Proposed · waiting for approval | A fix waits for a person on Approvals |
| Checking the fix worked | The fix ran; Polo is watching for a stable healthy period |
| Recovered after Polo’s verified fix | The checks stayed healthy after the verified change |
| Recovered on its own | Polo made no change, or the checks recovered before its fix ran |
| Recovered before Polo confirmed its fix worked | A fix had started, so it may or may not be why |
For a Grafana alert, the incident counts as recovered once Grafana resolves the alert. An alert that clears on its own does not show that its cause was fixed.
Investigations and fixes
Section titled “Investigations and fixes”An investigation uses only read tools. It reports what the checks or the alert show, the likely cause, and the next steps or access needed.
To let Polo propose a fix, tick it under Fixes Polo may propose on the monitor or alert source. The agent must have that change tool and be allowed to publish task reports. Each fix waits for approval under the workspace approval rules unless a standing approval covers that exact change. Polo proposes at most one fix per incident task, and counts the incident as fixed only after its checks stay healthy for a stable period.
Observability sources
Section titled “Observability sources”Investigations are only as good as what the agent can read. Connect these on Connections and tick them for the agent:
| Source | What the agent reads |
|---|---|
| Prometheus | Approved metric queries and availability objectives, directly or through a Grafana data source proxy |
| Loki logs | Recent log lines of each registered service, selected by exact stream labels. An agent can filter by one literal text; it cannot write its own query. |
| Tempo traces | Recent, slow or failed traces of each registered service, selected by its exact service name |
| Network diagnostics | DNS answers, certificate trust and expiry, and the HTTP status of a registered public origin |
Polo masks what looks like a secret in log lines before an agent reads them, and the log lines themselves are not shown in the task.