Fix a slow or failing pipeline

This guide follows the acceptance journey “The pipeline is too slow”. Polo measures a baseline, makes one specific change in a draft pull request, and compares that pull request’s CI with the baseline. Code, not the model, decides whether an improvement is established. The last section covers repairing failing CI on a pull request Polo opened.

Note

The whole journey has run end to end locally, with a scripted model and a local GitHub fixture. It has not been run against live GitHub. It works with GitHub Actions only.

Before you start

1. Connect the repository. Open Apps → Repositories and choose Connect a repository. Under GitHub Actions and pull request permissions:
   • tick Read workflow runs, job timings, and job logs;
   • list the workflow files under Approved write paths, so Polo can open a draft pull request;
   • tick Read CI for Polo-published pull requests.
2. Create an agent from the Pipeline agent template. See Build your first agent. It is set up to measure, edit the workflow, run the GitHub Actions lint and publish a draft pull request.

1. Assign the work

1. Open Home. In the Ask Polo box, choose the Improve a pipeline example or write your own request, naming the workflow.
2. Open Repository, environment, and budget and choose the Repository.
3. Choose Assign to Polo.

2. Polo measures a baseline

Polo selects the runs itself: the newest 20 completed push runs of one workflow from the last 30 days. It needs at least three successful ones.

The task shows a Push-run baseline card with the median wall time, each job’s median, the slowest step of each job, the first-attempt queue time, reruns and runner labels.

Wall time runs from the first job’s start to the last job’s end. It is not billed minutes.

3. Polo makes one change

Polo chooses one specific change from the slowest jobs and steps, edits the workflow file in its own workspace copy and lints it. It then proposes a draft pull request, which waits on Approvals. Approve it to publish the draft.

4. Polo compares the pull request’s CI

Polo does not start CI on the pull request. It reads the runs GitHub starts for the draft and compares them with the baseline. The comparison card gives one of these results:

Result	Meaning
Improvement established	Every pull request attempt finished faster than every baseline run, by more than the minimum Polo requires, and the checks passed.
Improvement not established	The attempts and the baseline overlap, the gain is too small, or more attempts are needed.
Slower than baseline	Every attempt finished slower than every baseline run.
PR checks did not pass	Speed is not compared as an improvement.
Waiting for CI	No completed attempt yet.
Not compared	For example, the pull request is no longer open.

An improvement needs at least two attempts and a change of at least 10 seconds or 5 %, whichever is larger. When the comparison asks for another attempt, Polo proposes a rerun of that workflow attempt, which an administrator approves.

5. Read the result

The task returns the draft pull request, a report with the measured medians, the limitations and how to roll back. When the evidence does not establish an improvement, the report says so: Polo checks its own speed claims against the newest comparison.

Merging the pull request is a separate step with its own approval.

Limits to know

• Polo does not see cache hits, the needs: graph, runner CPU and memory, or test flakiness.
• There is no measurement after the merge.
• Rollback is given as instructions; Polo does not perform it.
• If draft or path filters skip CI for the pull request, there is nothing to compare.

Repair failing CI on Polo’s own pull requests

When the checks of a draft pull request that Polo opened fail, Polo can fix them under one approval.

Caution

CI repair is implemented and tested against a local repository. It has not been verified against live GitHub.

To allow it:

1. In the repository’s connection, keep the approved write paths and Read CI for Polo-published pull requests switched on.
2. On the agent, switch on the two repair tools: the one that repairs failing CI on Polo’s pull request, and the one that adds a fix commit to it.

How it works:

1. Polo asks an administrator for one CI repair of that pull request. It appears on Approvals with the button Approve CI repair. Approving it writes nothing to GitHub.
2. For 60 minutes afterwards, Polo may add up to three fix commits to that draft without another approval. Each is recorded with the files it changed.
3. A fix commit that no repair covers, because none was approved, it expired or its three are used, waits for an administrator with Approve fix commit.

Polo adds fix commits only to its own draft. It is refused when the pull request is no longer Polo’s open draft or when anyone else pushed to the branch. Polo never pushes to a branch a person owns. There is one repair per pull request.

CI reliability report

For a view across the pipeline rather than one change, switch on Report CI reliability for the agent and ask, for example, “report CI reliability for the last 30 days”. The report shows the pull request failure rate, how many failures were the CI’s own fault, how long commits waited on CI and how long the protected branch stayed red. Add a schedule under Automation to get it every week.

Related

• Approvals and guardrails
• Capabilities