Pilot Scorecard: agree the pass line before an agent pilot starts

What you end up with

A pilot both sides agreed to judge the same way, and a result neither side can reargue.

What you get

• The method: write the pass line first, sign it, lock it, then let the numbers decide
• pilot.py: 8 terms checks (dates, measures with sources, targets better than baseline, refund rule, off switch, next steps, consent) and 4 results checks
• Sign-off refused inside an agent session, from anyone but the client owner, or if the terms changed; scoring refused if the terms changed after sign-off
• Case-study consent asked separately at sign-off, default no, and honored in the report
• AGENTS.md, CLAUDE.md, a Grok Bot standing order and a Hermes skill, a fictional example, and 28 tests

Who it is for

Providers who start with a trial, and owners who want to know in advance what would make it worth keeping.

Works with

Solid chips: the agent is named in this listing's own tools list. Dashed chip: One Python script, terms and results templates, a fictional dental practice example, and agent rules for each harness. Tested on Linux.

Tools needed

Cost to run: Free. Python is free.
Time to result: Terms ready to sign in about 20 minutes

How it works

1. Run the example

   Unpack the file, run python3 pilot.py sheet example/pilot.json out/, sign off at your own terminal with approve, then score example/results.json.

   Expected: PASS: 2 of 3 measures met, 2 needed.

2. Write your terms

   Fill in templates/pilot.json: one workflow, the dates, up to 5 measures with a baseline, target and source, and the next step either way.

   Expected: pilot.py check shows terms OK.

3. Sign and lock

   Send pilot-terms.md. The client owner signs off at their own terminal before day one.

   Expected: signoff.json exists; the pass lines are locked.

4. Score at the end

   Collect each number from its named source, fill results.json and run score.

   Expected: pilot-report.md with met or not met per measure and the agreed next step.

Sample prompts, free to use

Follow AGENTS.md. Confirm out/signoff.json exists, collect each measure from the source named in pilot.json, fill results.json with today's date, run pilot.py score and send the interim line. Do not edit pilot.json, sign off, or publish anything.

Free preview of the file

# Pilot Scorecard

**Agree what a passing pilot looks like before it starts. One page of terms the client owner signs, locked pass lines, and a scored report at the end that says pass or fail without an argument.**

Version 1.0, 2026-11-03. By PaperChaseWebb Inc. Free. Works with Grok Bot, Claude Code, Codex and Hermes, or by hand. Python 3.8 or newer, no installs.

## The problem

Most agent pilots end in a debate. The provider points at the hours saved, the client remembers the one bad reply, and nobody wrote down what "worked" meant. Worse, targets drift mid-pilot, so even a good result looks negotiated.

The fix is to write the pass line first: a few numbers, where each comes from, and what happens after a pass or a fail. The client owner signs it. Then the numbers decide.

## What you get out of it

- One terms file: one workflow, 30 days at most, up to 5 measures with today's number, a target and the source, plus data limits, an off switch, the fee and refund rule, and the next step either way.
- A one-page terms sheet the client owner reads and signs at their own terminal.
- Locked pass lines: if the terms change after sign-off, scoring is refused until the client owner signs again.
- A separate, default-no case-study question, so consent is never bundled into the deal.
- A report at the end: met or not met per measure, pass or fail, and the step both sides agreed to in advance.

## Who it is for

Providers and small agencies who offer a trial before a monthly plan. Ow

Worth knowing

• It scores what you give it. Pick sources both sides can see.
• A short pilot is a small sample. A pass says the workflow met the agreed lines in that window, nothing more.
• On 2026-11-03: 28 tests passed on Python 3.8, 3.9, 3.11, 3.12 and 3.13 on Linux, run from the files unpacked out of the pack. Agent rules guide; they were not loaded inside each harness for this release.
• Not a contract and not legal advice. Your own agreement governs the fee and refund.

Details

Version 1.0 · Updated Nov 3, 2026 · Markdown version of this page
License: Personal and commercial use. Download terms

Related

Prompt packFree

Agent Intake Packet: brief a new agent without handing over secrets

by PaperChaseWebb

A free kit for adding an agent to the ones you already run. Every agent that knows something writes one dated page in the same seven sections. A checker stops keys, passwords, stale pages, pasted chat logs and topics you never share. You sign off at your own terminal, and only then does the new agent get its standing context, with a limits card on top.

Works withClaude CodeCodexGrok BotHermesOne Python script, templates, a fictional example and agent rules for each harness. Tested on Linux

Solo business · Intermediate · 15 min setup

Prompt pack$29

Retainer Scope Sheet: itemize what an agent monthly covers

by PaperChaseWebb

One sheet per client that says line by line what your monthly agent retainer covers: the desks it runs and how many runs each, included hours and what happens over the limit, response times with covered hours, what is not included, what needs the client's yes, the reports they get and how changes are asked for. A checker fails every vague line ("ongoing support", "as needed", "ASAP"), an overage with no rate, a desk that sends or books with no yes row, and checks your proposals, change requests and invoices against the sheet. You approve each sheet at your own terminal.

Works withClaude CodeCodexGrok BotHermesPlain Markdown files, a small Python checker, and rules for AGENTS.md and CLAUDE.md, Claude Code and Codex notes, Hermes notes and a Grok Bot standing order. The agent files were checked by the checker on Linux; they were not loaded inside each agent for this release

Solo business · Intermediate · 30 min setup

Prompt packFree

Usage Cap Check: see the month coming before the agent bill does

by PaperChaseWebb

A free kit for providers who bill agent or model usage to clients, and for the owners who pay it. Write one plan with a monthly cap, alert points, a rule for going over and every job that spends. Copy usage from the provider's own export into a three-column log. A checker gives spend to date, the last 7 days' pace and where the month lands, flags jobs over budget, unplanned spend and jobs with no off switch, and turns each into one decision sheet the client owner answers in a single reply.

Works withClaude CodeCodexGrok BotHermesOne Python script, a plan and log template, a fictional plumbing company example, and agent rules for each harness. Tested on Linux

Solo business · Beginner · 15 min setup

Prompt pack$29

Workflow, Not Wrapper: test an AI product idea, shape the offer and set the price

by PaperChaseWebb

A prompt pack and method for anyone building or selling on top of a model. A seven-score absorption test, a one-page workflow product spec, a demo gap check, a problem mining loop, a five-talk buyer script, a channel translation table, a model column check, a true cost rebuild at three usage levels, an offer ladder with pricing rules, a price anchor audit, a check for copied success stories, a fidelity gate for generated output, a failure table, a worked example, an idea folder with a tested checker, a rules block and 19 prompts. What changed in v1.1: the demo gap check, buyer talks, true cost, price anchors and the checker are new.

Works withClaude CodeCodexClaudeGrok Bot+2Plain .md for any assistant, plus a skill file for Claude Code, Codex and Hermes, an AGENTS.md snippet and a Grok Bot rules block; the checker was tested on Linux with Python 3.13, the agent files were not loaded inside each harness

Solo business · Beginner · 45 min setup