# Release Watch: a per-change monitoring plan your agent writes from the diff, run by a fixed script

> After a change goes live, your agent reads the diff and writes a small watch plan for that change: which signals it could move, against which baseline, by how much, for how long. A fixed script runs the plan every five minutes, so no agent sits in a loop. On a trip, a second agent triages read-only and proposes one next step. Rollbacks, flag flips and alert changes stay with you.

- URL: https://paperchasewebb.ai/bots/marketplace/items/release-watch-per-change-monitoring-plan-from-the-diff/
- Type: Bot template
- Category: Vibe coding
- By: PaperChaseWebb
- Works with: Claude Code, Codex, Grok Bot, Hermes (named in the listing); A skill file for Claude Code, Codex and Hermes, an AGENTS.md snippet and a Grok Bot rules block; the runner and checker were tested on CSV metrics and a mock Prometheus, not a live production server, and the prompts were not run inside each harness
- Price: $24 USD
- Difficulty: Intermediate
- Setup time: 30 min
- Time to result: A first plan and a tested runner in about half an hour
- Format: Markdown (.md) file, release-watch-per-change-monitoring-plan-from-the-diff.md
- Version: 1.0, updated 2026-10-28
- License: Use in your own repos and on installs you do for clients and teams. Do not resell or republish the pack itself.

## What you end up with

Each change gets its own watch tied to the paths it touched, a silent traffic drop trips as reliably as an error spike, and nothing is rolled back until you have read the triage and said yes.

## What you get

- 18 sections in one .md file
- watch_run.py: increase, above and drop rules with persist, floors and minimum samples; CSV and Prometheus sources, GET only; exit 1 only on a trip, exit 2 on bad input
- A plan prompt that writes the watch from the diff and a read-only triage prompt that proposes and stops
- A skill file for Claude Code, Codex and Hermes, an AGENTS.md snippet, a Grok Bot rules block and timer setup
- Owner approvals in approvals.csv tied to the plan's sha256
- watch_check.py: fifteen read-only checks, with a fixture builder and the expected output
- A labeled eval set of six cases to count false alarms and misses before you trust it
- Tests: a runner selftest, a mock Prometheus test and one test_all.sh
- Ten prompts and a failure table
- A look at eight canary skills, rollback skills and runbooks, with sources and credit for the idea

## Who it is for

Builders and small teams who merge many changes a day with help from agents, have metrics somewhere, and want each change watched without paying an agent to stare at dashboards.

## Tools needed

- Claude Code, Codex, Hermes or Grok Bot: reads the diff and writes the watch plan; a fresh read-only session triages a trip
- Python 3: runs the watch runner, the read-only checker, the tests and the eval set; standard library only
- A scheduler: cron, a CI schedule or your harness's scheduler runs the runner every five minutes, so no agent stays in a loop
- Your metrics: a Prometheus server read through query_range with GET only, or any metrics exported to CSV
- You need: Claude Code, Codex, Hermes or Grok Bot
- You need: Python 3.9 or newer
- You need: Read-only access to Prometheus, or metrics you can export to CSV
- You need: A scheduler: cron, CI or your harness's own

Cost to run: The pack is the only purchase. The runner uses no model tokens; the agent is used only to write a plan and to triage a trip.

## How it works

1. **Run the tests.** Copy the release-watch folder into your repo and run sh test_all.sh. Expected: 14 ok and 3 ok with 0 failures, bad plan exits 2, 15 findings on seeded and 0 on fixed, eval caught 3, missed 0, false alarms 0.
2. **List your sources.** Have your agent draft WATCH-SOURCES.md from your dashboards with prompt 10: metric name, source, unit and what normal looks like. Expected: A short list you have read and corrected.
3. **Write the first plan.** After your next merge, run the plan prompt. It runs watch_check.py until 0 findings and shows you the plan and its sha256. Expected: A plan where every touched path is covered and every signal quotes the line it protects.
4. **Approve and run.** Add the approvals.csv row with the hash, then start the runner on a five-minute timer. Expected: runlog.jsonl grows by one line per run and report.md shows WATCHING, then CLEAR or TRIPPED.

## Sample prompt: Write the plan (Grok Bot, Claude Code, Codex, Hermes)

```
Read the diff for this change and write the watch plan with PLAN-PROMPT.md. Run watch_check.py until it shows 0 findings.
Show me the plan and its sha256, then stop. Do not start the watch.
```

Prompt 1 of 10. The owner adds the approval row.

## Sample prompt: Triage a trip (Grok Bot, Claude Code, Codex, Hermes)

```
The watch tripped. Run TRIAGE.md read-only and stop after writing triage.md.
Propose one next step. Do not roll back, flip a flag or change an alert.
```

Prompt 5 of 10. Run it in a session with read-only tools.

## Free preview of release-watch-per-change-monitoring-plan-from-the-diff.md

````markdown
# Release Watch: a per-change monitoring plan your agent writes from the diff, run by a fixed script

Version 1.0, 2026-10-28. By PaperChaseWebb. Works with Grok Bot, Claude Code, Codex and Hermes. Plain Markdown prompts, a skill file, a rules block, an AGENTS.md snippet, a runner, a read-only checker and a labeled eval set, all Python 3 standard library. Tested as described in section 11: everything ran on Linux with Python 3.13.5 against CSV metrics and a mock Prometheus server on 127.0.0.1. It was not run against a live production Prometheus, and the prompts were not run inside each harness for this release. Both are marked where they matter.

A change merges. Dashboards say nothing in particular. Two hours later someone notices checkout traffic is down a third, and nobody can say which of the eleven deploys that afternoon did it. The usual fixes do not help much. A wall of global alerts pages on everything or nothing. A post-deploy checklist gets skipped on a busy day. An agent told to "watch production" keeps a session open, reads the same graphs every minute, spends tokens doing it, and is free to type a rollback command when it gets nervous.

Release Watch splits the job three ways. An agent reads the diff and writes a small plan for that one change: which signals the change could move, against which baseline, by how much, for how long. A fixed script runs the plan on a timer, so no agent sits in a loop. When a signal trips, a second agent triages read-only and writes 
````

## Worth knowing

- The runner was tested on CSV metrics and on a mock Prometheus query_range server on 127.0.0.1, not on a live production Prometheus. Try one plan on a quiet change before you rely on it.
- The plan and triage prompts and the skill file were not run inside Claude Code, Codex, Hermes or Grok Bot for this release. Read the first plan your agent writes before you approve it.
- This is not a replacement for global alerts on outages that have nothing to do with a change.
- On 2026-10-28, on Linux with Python 3.13.5, the runner selftest gave 14 ok, the mock Prometheus test 3 ok, both with 0 failures; a plan with no timezone exited 2; watch_check.py reported the fifteen expected findings on the seeded fixture and 0 on the fixed one, with files unchanged and exit 2 on a missing folder; the eval set gave caught 3, missed 0, false alarms 0, quiet and right 3.

## Questions

**Why not let the agent watch the dashboards itself?** An agent in a loop costs tokens every minute, can give different answers on the same data, and can be talked into a rollback. A fixed script gives the same answer every time, costs nothing to run, and only wakes an agent when something trips.

**Will it roll back for me?** No, on purpose. The report names the rollback command as a proposal, triage writes up the evidence, and you decide. The checker flags any plan that sets a rollback or action to run automatically.

**Do I need Prometheus?** No. The runner also reads a CSV with time, metric and value columns, so any tool that can export metrics works.

**How do I know it is not too noisy?** Run the eval set, then add your own past incidents and quiet releases as labeled cases. It prints caught, missed and false alarms, so you tune with numbers.

## How to buy

$24 one time at https://paperchasewebb.ai/bots/marketplace/items/release-watch-per-change-monitoring-plan-from-the-diff/. Stripe checkout, then an instant download page; the download link is also in your emailed receipt.

Seller: PaperChaseWebb Inc., mgmt@paperchasewebb.com