Fix PRcheckout/helm #481
needs-human- replicaCount: 2
+ replicaCount: 4
- timeoutSeconds: 2
+ timeoutSeconds: 5
Tier-0 On-call agent
Takes the page first.
A page comes in. Outguard writes a notebook, then opens a Fix PR against the system and a Hygiene PR against the monitor. Git is the only action channel.
ClassificationReal · SLO burn 8.2×
Evidence notebook ↗Hygiene PRinfra/alerts #2291
checks running- for: 1m
+ for: 10m
+ page_if: burn_1h > 6
Policy: fix needs one owner · hygiene after checksNo live mutation
Git-only actionPolicy in gitNotebook before diffVerdict after merge
01 Doctrine
We do not talk to production.
We propose a change to the git that already is production, and a change to the monitor that lied about it.
Investigation exists to justify a small diff. Owners inspect it, merge it, or revert it. Live kubectl, cloud-console applies, and RCA chatbots are out of scope.
02 The loop
The work is a merge.
Classify the page, write the notebook, open the PRs, then score whether it helped.
- 01 / TriageClassify first.
Real, flappy, duplicate, SLO-not-burning, or no owner. That happens before anyone starts clicking dashboards.
- 02 / EvidenceThe notebook is the case.
Queries, hypotheses, last deploy, timestamps. Append-only. Inconclusive is a valid stop.
- 03 / ChangeFix PR and Hygiene PR.
One diff for the system that caused the page. One for the monitor, so the same threshold does not fire tomorrow.
- 04 / LearnScore the merge.
Burn, silence, reverts. A miss or a revert makes the next episode on that service stricter.
03 Two pull requests
Two PRs, different jobs.
The Fix planner is not the hygiene planner. Different diffs, different merge rules.
A
Fix PR · remediation as git
The system underneath.
Small config and Helm diffs, with the notebook and a rollback plan attached. In the MVP these always wait for an owner.
- Timeouts, replicas, flags, cache TTLs
- Evidence and similar past verdicts
- Always
needs-humanin the MVP
B
Hygiene PR · monitor as code
The monitor that lied.
The threshold, the burn gate, the owner label, or the duplicate that should never have paged. Smaller than a Fix PR, so policy can auto-merge after checks.
- Duration and SLO burn gating
- Routing, labels, and expiring silences
- Smaller diffs, eligible for policy auto-merge
04 Guardrails
Policy is a file in git.
CODEOWNERS and CI stay in charge. The model does not override forbid.
outguard.yaml
Auto-merge, needs-human, forbid. Lives in the customer repo, not only in the SaaS.
Notebook first
Queries and hypotheses land in an append-only notebook before any PR exists.
Git is the action channel
CODEOWNERS and CI remain in control. Restarts become Runbook Actions in git, never hidden mutations.
Verdict after merge
Burn, silence, reverts. A miss tightens the next episode on that service.
05 FAQ
What Outguard is not
It opens PRs. It is not the pager, not a metrics vendor, and not a chatbot talking to the cluster.
What is Outguard?
A Tier-0 on-call agent for infrastructure teams. It takes a page and opens two git PRs: a Fix PR that changes the system, and a Hygiene PR that changes the monitor.
Does Outguard replace PagerDuty or Datadog?
No. It is not the pager and not an observability vendor. It sits beside PagerDuty, Grafana IRM, or Alertmanager, and beside Datadog or Prometheus.
Does Outguard mutate production?
No. Git is the only action channel. No live kubectl, no cloud console, no SSH. If the fix is a restart, the PR adds or invokes a Runbook Action in git.
What is a Fix PR?
A change to the system that caused the page: config, Helm values, timeouts, replica count. It includes notebook evidence, predicted blast radius, owners, and a rollback plan. In the MVP, Fix PRs are needs-human.
What is a Hygiene PR?
A change to the monitor that fired, so the same bad page does not repeat. Duration gates, SLO burn gates, routing, labels, an expiring silence, or deleting a duplicate.
Who decides what can merge?
outguard.yaml in the customer git. The validator enforces auto-merge, needs-human, and forbid in code. The LLM cannot override forbid or live.mutate.
What happens after a merge?
A Watcher scores the episode: did burn fall, did the monitor stay quiet, was the PR reverted. That Verdict makes the next episode on the service stricter after a miss or a revert.
06 Design partners
If the monitors are already in git.
On-call that has seen the same page more than once.
