Evidence Standard: what counts as proof for each kind of agent work, who checks it, and what is refused
by PaperChaseWebb
Agents report done from the plan as often as from the result. Five plain files write down, for each class of work, the proof that counts, who checks it (never the producer), what is refused and how long proof is kept. Agents log each claim with its evidence, seal it by sha256 and report in the words evidence.py say allows: done, partly done with the gap named, waiting on the owner, or reported, not verified. The agent's own summary is never proof. Fifteen read-only checks run on the standard and the claims ledger.
- Price
- $24
- Difficulty
- Intermediate
- Setup
- 30 min
- Format
- .md file
evidence-standard-what-counts-as-proof-for-agent-work.md · 98 KB · instant download after checkout
Want this built for you instead? We can build it for you.
What you end up with
A written rule for what your agents may call done, checked before they report: every claim tied to evidence someone else can open, checked by someone other than the agent that did the work, with people, money, records and releases going to a person and your yes.
What you get
- 19 sections in one .md file
- The standard folder: standard.json, classes, proof kinds, refusals and a claims ledger, with blank templates
- Eight proof kinds with the agent's own narration listed and refused, three checker roles, two gates and a three-level claim ladder with the words for each
- evidence.py: seal evidence by sha256, say (exit codes 0, 10, 11, 12), period reports, owner approval tied to a sha256, verify, and one use per yes
- evidence_check.py: fifteen read-only checks, with a seeded test standard (15 findings, then 0)
- A skill file for Claude Code, Codex and Hermes, an AGENTS.md snippet and a Grok Bot rules block
- Ten prompts, a failure table and a worked example
- A look at completion receipt packs, verification skills and transcript auditors, with sources
Who it is for
Anyone who lets agents do real work (code, research, copy, outreach drafts, record edits, releases) and wants every done in their reports backed by evidence a second party can open.
Works with
Solid chips: the agent is named in this listing's own tools list. Dashed chip: A skill file for Claude Code, Codex and Hermes, an AGENTS.md snippet and a Grok Bot rules block; the scripts were tested on Linux with Python 3.13, the agent files were not loaded inside each harness.
Tools needed
Cost to run: The pack is the only purchase. The scripts use no model tokens.
Time to result: A ratified standard for five to eight classes of work in an afternoon
How it works
Run the tests
Copy the evidence-standard folder and run sh test_all.sh.
Expected: 15 findings on the seeded standard and 0 on the fixed one, 15 of 15 single faults caught alone, checker tests 38 passed, evidence.py tests 52 passed, exit 2 on a missing folder.
Write your standard
Run evidence.py init, use prompts 1 to 3 to find the classes of work your agents already do, and write the proof, checker, gate and refusals for each. Run evidence_check.py until only E15 remains.
Expected: Every class names what counts as proof, a checker who is not the producer, what is refused and where proof is kept.
Ratify and connect
The owner runs evidence.py approve DIR --standard at their own terminal. Install the skill file, paste the AGENTS.md snippet, or paste the rules block into Grok Bot.
Expected: Each agent logs and seals a claim before reporting and uses the words evidence.py say gives.
Live with it
A fresh session or a named person checks each claim. The owner says yes to gated claims and approves the period report by hash. Review the standard monthly with prompt 9.
Expected: verify passes only for an unchanged report under the current standard and an unchanged claims ledger, within approval_hours, once.
Sample prompts, free to use
Before you tell me anything is done, add the claim to claims.csv with the evidence and kinds, run evidence.py seal, leave checked_by empty, and run evidence.py say. Report using its words.
Prompt 4 of 10.
You did not do this work. Open each piece of evidence listed for claim <ID>. Say whether it shows what the claim says, after the last change, and quote the line or output that does. If it does not, say what is missing. Do not fix anything.
Prompt 5 of 10.
Free preview of the file
# Evidence Standard: what counts as proof for each kind of agent work, who checks it, and what is refused Version 1.0, 2026-10-29. By PaperChaseWebb. A written evidence standard as five plain files, a script your agents use to seal evidence and to get the words their evidence allows, a read-only checker with fifteen checks, templates, a seeded test standard, ten prompts and the tests, all Python 3 standard library. Works with Claude Code, Codex and Hermes through a skill file and an AGENTS.md snippet, and with Grok Bot through a rules block. Tested as described in section 11: everything ran on Linux with Python 3.13, including owner approvals typed at a pseudo-terminal. The skill file, snippet and rules block were not loaded inside the harnesses for this release, and the pack has not been used on a real team's work. Each is marked where it matters. Agents report work in confident sentences. "Fixed the rounding bug, all tests pass." "Merged the duplicate supplier rows." "Version 2.4 is live." Some of those are true. Some come from the plan rather than the result: the test was a stub, the run happened before the last edit, the tool returned a 200 and the record is still wrong, the page built and nobody loaded it. The person reading the summary cannot tell which is which without redoing the work, and that is the job they handed off. The fix is a standard written before the work, not a better prompt after it. For each class of work you name what counts as proof, who checks it,
Test record
- Date
- Oct 29, 2026
- What was run
- evidence_check.py on a seeded standard and a fixed copy, each planted fault alone, a missing input, and the unit tests. Linux, Python 3.13.
- Result
- 15 findings on the seeded standard, 0 on the fixed one; 15 of 15 planted faults caught alone; exit 2 on missing input; nothing written. 38 checker and 52 evidence.py tests passed, 0 failed.
- Not run
- The agent files inside Claude Code, Codex, Hermes or Grok Bot. A real team's work. Windows.
Copied from this item's dated test notes. What was not run stays in.
Worth knowing
- evidence.py answers questions; it cannot stop an agent that never asks, or one that writes a false file and seals it. That is why a separate checker opens the evidence. This is not a sandbox.
- The skill file, AGENTS.md snippet and Grok Bot rules block were not loaded inside Claude Code, Codex, Hermes or Grok Bot for this release, so how reliably each harness logs claims before reporting was not measured. The pack has not been used on a real team's work, and there is no claim about how many false done reports it prevents. The test standard is made up. Linux only; Windows was not tested.
- It checks that a claim is backed by evidence a second party can open, not whether the work is good. Not a compliance certification (SOC 2, ISO 42001, EU AI Act).
- The approval step refuses piped input and the environment variables Claude Code, Codex, Hermes and Grok Bot set. That stops an agent approving by mistake; it is not a lock against someone with the same account.
- On 2026-10-29, on Linux with Python 3.13: evidence_check.py gave 15 findings on the seeded standard (each check once) and 0 on the fixed one, caught 15 of 15 single faults with only its own check, wrote nothing, and exited 2 on a missing folder, missing claims.csv or broken standard.json; checker tests 38 passed, 0 failed; evidence.py tests at a pseudo-terminal 52 passed, 0 failed.
Questions
How is this different from a verification-before-completion skill?
Those skills tell the agent to check before it says done, and they are a good rule. Here the rule covers every class of work, not just code, names a checker who is never the producer, lists what is refused, and the standard itself is checked fifteen ways.
What does the agent say if the evidence is missing?
evidence.py say answers reported, not verified (exit 12). If some evidence is there and the gap is written down, it answers partly done with the gap named (exit 10).
Can a second agent check outreach or a release?
No. Classes that touch people, clients, money, records or a release are checked by a person or the owner and need the owner's yes, tied to that claim's sha256.
What if someone edits the evidence after the claim?
The sealed sha256 stops matching, the checker flags it and say drops the claim to reported, not verified until it is sealed and checked again.
Details
Version 1.0 · Updated Oct 29, 2026 · Markdown version of this page
License: Use on your own work and on standards you set up for clients and teams. Do not resell or republish the pack itself. Download terms
Related
Evidence Standard: five rules and a five-check script (free sample)
From the Evidence Standard: five rules for what your agents may call done (a person owns the standard, every class of work names its proof and its checker, the agent's own account is never proof, people, money, records and releases are checked by a person, nobody says done on their own work), and a read-only script that checks one small file for them.
Rubric Library: name, version and own the scoring rubrics hiding in workflows
Every score, rank, flag or triage step an agent runs has a rubric in it, usually unnamed and rewritten each time. Six plain files give each rubric an owner, a version, grades with definitions and actions, evidence rules, borderline examples and an adversarial set. Agents load a rubric by name and number with rubric.py and check every score; unnamed or default is unverdicted, and anything touching a person, a client or money goes to a person. Fifteen read-only checks run on the library itself.
Production Readiness Audit: decide from evidence whether an agent system can go live
Twelve areas, each pass, gap or na, and every pass points to a dated evidence file. Gaps get a named owner and a due date, five areas block go-live until fixed, drills are run rather than described, and the verdict is signed by someone who did not build the system. A fifteen-check script reads the folder, the report is sealed to a hash of the evidence, and the owner's go-live yes is tied to the report's sha256.
Research Contract for Agents: claims, sources and dates your agent must keep
A contract your agent follows on every research run, kept as plain files: each claim with its source, date and confidence, primary sources before secondary, quotes word for word with a location, contradictions listed, unknown allowed, and no citation to anything it did not open. A fifteen-check script reads the folder and names every break in a plain sentence, and nothing is published or acted on until the owner approves the exact report by its sha256.
Two-Agent Build and Test Loop for paid apps: one agent builds, another tests, nothing passes on its own say-so, and side effects happen once
A tested setup for apps that take money: one agent writes the code, a separate tester session writes the stories and tests and gives the verdicts, a small runner records every red, green and revert exit code itself, and the owner says yes to the plan, any payment or pricing work, each merge and each release. Version 1.1 makes side effects happen once: one idempotency key per refund, charge, email or deploy, checked against a ledger before acting, the original event saved next to the run it started, a retry budget that stops and asks the owner, and every failure saved as a regression test. Two read-only checks with fifteen checks each audit the finished run.
Check the Claim (expanded edition): a method, prompts and a skill for checking AI claims
Make your agent check income, tool, popularity, quote, news and offer claims before they reach a brief, a pitch, a purchase or a post: label how much was read, run the creator's own arithmetic, trace sources back to first hand, and give each claim one of five verdicts on a card the owner approves. v1.1 adds an evidence level label (full, caption, summary or wall), four new arithmetic tests, a section on tracing a source back to first hand, screens for offers, quotes and slogans, four more red flags, a sixth worked example, a claim folder with a fourteen-check read-only checker and an owner approval helper, agent files for Grok Bot, Claude Code, Codex and Hermes, and twelve new prompts (26 in all). Price unchanged.
Agent Readiness Self-Check: score one workflow before you hand it to an AI agent
A free self-check for agencies and small businesses. Name the decision, then score one workflow across eight areas (workflow mapped, data access, approval gates, owner per agent, handoffs, cost tracking, security and vetting, measurement) with an evidence rule, three gate rules and a skeptical pass, and read your band and the next steps it allows. v1.1 starts with the decision the score will change, scores unknowns as unknown, adds rules for conflicting facts and lookalike public records, ties every band to the next steps it allows with the Production Readiness Audit before go-live, makes owner approval concrete (prepared and approved status, a one-reply approval line, decisions the agent must leave open, a done test), adds agent files for Grok Bot, Claude Code, Codex and Hermes, a second worked example (missed call to booked job), a readiness folder with a fifteen-check read-only checker, a front door test with four questions to ask anyone selling you an agent install, and a plain note on how it was tested. Still free.
Model Provenance Record: where an open-weight model came from, how it behaves, and who said yes, before it is installed
Before any open-weight model goes on a business machine, fill one record: the publisher and exact revision, the license and whether it allows your use, every file with its published and local sha256, a header check that the format cannot run code (or a recorded scan if it can), the lineage back to the original, and a 15-test guardrail behaviour test that must still refuse clear misuse and still answer ordinary questions. Each result is tied to the hash of the weights and the test text, judged by someone who did not run it, and the owner says yes to one install hash. Fifteen read-only checks run on the record and the files.
Source Shelf: a short, owner-approved list of industry reports your team reads, and a way to read them
Keep a small set of major industry reports as your team's default evidence base: each with its edition, publish date, link on the publisher's own site, the date someone last looked for a newer edition, and a note on what the publisher sells. Read any one of them against three questions (what is changing, who is struggling, what could help them), with word-for-word quotes and page numbers, the third answer marked as your inference, and every figure matched to its quote. Every add, drop and replace waits for the owner's yes. Fifteen read-only checks run on the folder.
Replicate Before You React: a trial kit for claims about AI agents
Test a claim about a coding agent on your own setup before you change anything: a security hole, a token-saving prompt, a hidden cost, a skill that makes the agent better. Restate the claim, run each setup 10 to 20 times in fresh containers with one capped key, use a harmless canary instead of a real payload, run a control arm for cost and quality, and get rates with honest intervals and a result card that says what you did not test.
Startup Credits Sweep: an eligibility matrix, honest applications and an expiry ledger
Sweep startup programs for cloud, AI API and SaaS credits from your own facts. Each fact names its record; each program is read on its own page with the date; each requirement is marked meets, fails or unknown against one fact, and the matrix sets the verdict. Application drafts cite every fact and say it word for word, so nothing is stretched or invented to fit a form. A ledger counts credits only after the approval notice and reminds you before each grant expires. Every application waits for the owner's yes, and the owner submits it by hand. Fifteen read-only checks run on the folder.