Production Readiness Audit: decide from evidence whether an agent system can go live
by PaperChaseWebb
Twelve areas, each pass, gap or na, and every pass points to a dated evidence file. Gaps get a named owner and a due date, five areas block go-live until fixed, drills are run rather than described, and the verdict is signed by someone who did not build the system. A fifteen-check script reads the folder, the report is sealed to a hash of the evidence, and the owner's go-live yes is tied to the report's sha256.
- Price
- $39
- Difficulty
- Intermediate
- Setup
- 30 min
- Format
- .md file
production-readiness-audit-evidence-before-go-live.md · 78 KB · instant download after checkout
Want this built for you instead? We can build it for you.
What you end up with
A go-live verdict you can defend: every pass backed by a file, every gap owned and dated, rollback, pause and refusals tried for real, and a signed report that shows if anything changed after it was written.
What you get
- 19 sections in one .md file
- The audit folder: audit.json, twelve areas, evidence, gaps, drills, secrets, access, monitors, report and approvals, with blank templates
- What counts as evidence for each area, and what does not
- audit.py: set up a folder, seal the report to its evidence, owner approval tied to the report sha256, and verify
- audit_check.py: fifteen read-only checks, with a seeded test audit (15 findings, then 0)
- A skill file for Claude Code, Codex and Hermes, an AGENTS.md snippet and a Grok Bot rules block
- Ten prompts for the builder, the auditor and the owner, a failure table and a worked example
- A look at eleven readiness checklists and kits, with sources
Who it is for
Anyone about to let an agent workflow touch real customers or real money, and anyone who installed one a while ago and is not sure it would hold up on a bad day.
Works with
Solid chips: the agent is named in this listing's own tools list. Dashed chip: A skill file for Claude Code, Codex and Hermes, an AGENTS.md snippet and a Grok Bot rules block; the scripts were tested on Linux with Python 3.13, the agent files were not loaded inside each harness.
Tools needed
Cost to run: The pack is the only purchase. The checker and approval script use no model tokens; drills use whatever your agent normally costs, on a small cap.
Time to result: A sealed, signed audit for one system in about a day of work spread over a week
How it works
Run the tests
Copy the production-readiness-audit folder and run sh test_all.sh.
Expected: 15 findings on the seeded audit and 0 on the fixed one, 15 of 15 single faults caught alone, checker tests 24 passed, approval tests 22 passed, exit 2 on a missing folder.
Add the rules to your agent
Install the skill file for Claude Code, Codex or Hermes, paste the AGENTS.md snippet, or paste the rules block into Grok Bot.
Expected: The agent starts with audit.py init and asks who the builder and the auditor are.
Gather evidence and run the drills
Use prompts 1 to 7. The owner runs the pause drill. Run audit_check.py until only R14 and R15 remain.
Expected: Every pass points to a dated file and every gap has an owner and a due date.
Audit, seal and approve
The auditor writes report.md and runs audit.py seal. The owner reads it and runs audit.py approve at their own terminal. The agent runs audit.py verify before going live.
Expected: verify prints approved for this report and these inputs; any later edit needs a new seal or a new yes.
Sample prompts, free to use
Create an audit folder for <system> with audit.py init. Ask me for the owner, the builder, the auditor and the scope, and write audit.json. If you helped build <system>, put yourself down as the builder. Go through the twelve areas in areas.csv. For each, find a file that proves it, copy it into evidence/ with the date, and mark pass. Where you cannot find proof, mark gap and say what is missing.
Prompts 1 and 2 of 10.
Read the folder from scratch and open every evidence file cited. Mark anything weak as a gap. Write report.md with Verdict, Blocking gaps, Exceptions and Signed, choosing the strictest verdict the evidence supports. Run audit.py seal, then audit_check.py until only R15 remains. Give the owner the report hash and stop.
Prompt 8 of 10.
Free preview of the file
# Production Readiness Audit: decide from evidence whether an agent system can go live Version 1.0, 2026-10-28. By PaperChaseWebb. One audit folder of plain files, twelve areas, a read-only checker with fifteen checks, a script that seals the report to its evidence and records the owner's go-live yes, templates, a seeded test audit, ten prompts and the tests, all Python 3 standard library. Works with Claude Code, Codex and Hermes through a skill file and an AGENTS.md snippet, and with Grok Bot through a rules block. Tested as described in section 11: everything ran on Linux with Python 3.13, including owner approvals typed at a pseudo-terminal. The skill file, snippet and rules block were not loaded inside the harnesses for this release, and the pack has not been used on a real client system. Each is marked where it matters. Most production readiness checklists are long lists of good questions. The trouble starts after the boxes are ticked. A box says "rollback plan: yes" and nobody has rolled back. The secrets row is ticked while an old key sits in the repo history. The person who built the agent fills in the list and signs it. Three months later the list is still "passed" and half of it is no longer true. This pack keeps the questions short and asks for proof instead. Each of the twelve areas is pass, gap or na, and a pass has to point to a dated file in the audit folder: a config excerpt, a log sample, test output, a drill note. Gaps get a named owner and a due date. Fi
Worth knowing
- The checker proves that each pass has a dated, recent evidence file that was not changed after the report, and that the verdict fits the open gaps. It cannot prove an evidence file is true or a drill really passed; the auditor reads them.
- The skill file, AGENTS.md snippet and Grok Bot rules block were not loaded inside Claude Code, Codex, Hermes or Grok Bot for this release, and the pack has not been used on a real client system. The test audit is made up. Linux only; Windows was not tested.
- This is not a compliance certification (SOC 2, ISO 27001, EU AI Act) and not a model quality evaluation.
- The approval script refuses piped input and the environment variables Claude Code, Codex, Hermes and Grok Bot set. That stops an agent approving by mistake; it is not a lock against someone with the same account.
- On 2026-10-28, on Linux with Python 3.13.5: audit_check.py gave 15 findings on the seeded audit (each check once) and 0 on the fixed one, caught 15 of 15 single faults with only its own check, wrote nothing, and exited 2 on a missing folder, a missing audit.json, areas.csv, drills.csv or report.md, or broken audit.json; checker tests 24 passed, 0 failed; approval tests at a pseudo-terminal 22 passed, 0 failed.
Questions
How is this different from the Agency Install Kit?
The Install Binder in the kit records the decisions made while building one workflow, and it is written by the builder. This audit comes after: it asks for proof that the built system holds up, and it is signed by someone else. Use the binder to install and this to decide go-live.
Why not a score?
A score can pass with a leaked key or an untested rollback still open. Here five areas block go-live until fixed, and the verdict can never be softer than the open gaps allow.
Can an agent be the auditor?
Yes, if it had no part in the build. A second agent that reads the folder from scratch works. The agent that built the system never signs its own verdict.
How often should I rerun it?
Every 90 days, or after a model change, a new connector, a new action or an incident. Evidence and the rollback drill stop counting after 90 days, so the checker will tell you.
Details
Version 1.0 · Updated Oct 28, 2026 · Markdown version of this page
License: Use on your own systems and on audits you do for clients and teams. Do not resell or republish the pack itself. Download terms
Related
Production Readiness Audit: five rules and a five-check script (free sample)
From the Production Readiness Audit: five rules for going live (a pass needs evidence, every gap has an owner and a due date, roll back once on purpose, test refusals, someone who did not build it signs), and a read-only script that checks one audit file for them.
Agency Install Kit: install one agent workflow in two weeks, with a binder a script can check
Five paid packs and the free Readiness Self-Check, tied into one two-week install by a new Install Binder: plain files for readiness, one workflow, agent owners and tested pauses, a gate register, a reach inventory, a cost cap, a dry run with refusal cases, a scorecard with kill rules and a run sheet with an off switch. A fifteen-check script names every missing decision, and the owner's go-live yes is tied to the binder's sha256.
Agency AI Ops Workbook: the Install sprint, self-serve
A two-week workbook for owner-led agencies and small businesses: install one AI agent workflow yourself with the same steps as our $5,000 Install sprint. Discovery and mapping, picking the first workflow from your Self-Check score, an agent card with an owner and a tested pause, data access and a test set, an approval gate register, handoff packets, cost limits, tool vetting, a baseline and stop rule, a dry run, a go-live checklist and a 30-day review. Ten worksheets, 13 prompts with harness notes.
Release Watch: a per-change monitoring plan your agent writes from the diff, run by a fixed script
After a change goes live, your agent reads the diff and writes a small watch plan for that change: which signals it could move, against which baseline, by how much, for how long. A fixed script runs the plan every five minutes, so no agent sits in a loop. On a trip, a second agent triages read-only and proposes one next step. Rollbacks, flag flips and alert changes stay with you.
Shrink the Attack Surface: patch with proof
Your agent lists everything your app exposes, read-only: routes, open ports, public buckets, secrets in the repo, unused dependencies, debug endpoints, admin paths, CORS, API key scopes and its own tool permissions. You decide keep, remove or lock for each. Every fix gets a proof that fails before the change and passes after it, and the proof stays as a regression test. Red-team drills then check that the agent still asks before risky steps, stops retrying broken tools and never charges or emails twice.
Agent Readiness Self-Check: score one workflow before you hand it to an AI agent
A free self-check for agencies and small businesses. Score one workflow across eight areas (workflow mapped, data access, approval gates, owner per agent, handoffs, cost tracking, security and vetting, measurement) with an evidence rule and three gate rules, read your band and what to do next, plus an interview prompt for Grok Bot, Claude Code, Codex or Hermes, a score sheet and a worked example.