# Pilot Scorecard: agree the pass line before an agent pilot starts

> A free kit for providers who offer a trial before a monthly plan, and for owners who want to try one. Write one page of terms: one workflow, 30 days at most, up to 5 measures with today's number, a target and where each number comes from, plus data limits, an off switch, the refund rule and the next step after a pass or a fail. The client owner signs it at their own terminal, the pass lines lock, and the end report says pass or fail per measure without an argument.

- URL: https://paperchasewebb.ai/bots/marketplace/items/pilot-scorecard-agree-the-pass-line-before-you-start/
- Type: Prompt pack
- Category: Solo business
- By: PaperChaseWebb
- Works with: Claude Code, Codex, Grok Bot, Hermes (named in the listing); One Python script, terms and results templates, a fictional dental practice example, and agent rules for each harness. Tested on Linux
- Price: Free USD
- Difficulty: Beginner
- Setup time: 15 min
- Time to result: Terms ready to sign in about 20 minutes
- Format: Markdown (.md) file, pilot-scorecard-agree-the-pass-line-before-you-start.md
- Version: 1.0, updated 2026-11-03
- License: Personal and commercial use.

## What you end up with

A pilot both sides agreed to judge the same way, and a result neither side can reargue.

## What you get

- The method: write the pass line first, sign it, lock it, then let the numbers decide
- pilot.py: 8 terms checks (dates, measures with sources, targets better than baseline, refund rule, off switch, next steps, consent) and 4 results checks
- Sign-off refused inside an agent session, from anyone but the client owner, or if the terms changed; scoring refused if the terms changed after sign-off
- Case-study consent asked separately at sign-off, default no, and honored in the report
- AGENTS.md, CLAUDE.md, a Grok Bot standing order and a Hermes skill, a fictional example, and 28 tests

## Who it is for

Providers who start with a trial, and owners who want to know in advance what would make it worth keeping.

## Tools needed

- Python 3.8 or newer: pilot.py: terms check, terms sheet, owner sign-off with separate case-study consent, and scoring against the locked lines; standard library only
- Grok Bot, Claude Code, Codex or Hermes (optional): drafts the terms, collects each number from its named source and runs the score under AGENTS.md; never signs off, moves a target or publishes
- You need: Python 3.8 or newer
- You need: One workflow to pilot and a baseline number for each measure

Cost to run: Free. Python is free.

## How it works

1. **Run the example.** Unpack the file, run python3 pilot.py sheet example/pilot.json out/, sign off at your own terminal with approve, then score example/results.json. Expected: PASS: 2 of 3 measures met, 2 needed.
2. **Write your terms.** Fill in templates/pilot.json: one workflow, the dates, up to 5 measures with a baseline, target and source, and the next step either way. Expected: pilot.py check shows terms OK.
3. **Sign and lock.** Send pilot-terms.md. The client owner signs off at their own terminal before day one. Expected: signoff.json exists; the pass lines are locked.
4. **Score at the end.** Collect each number from its named source, fill results.json and run score. Expected: pilot-report.md with met or not met per measure and the agreed next step.

## Sample prompt: Weekly pilot score (Grok Bot, Claude Code, Codex, Hermes)

```
Follow AGENTS.md. Confirm out/signoff.json exists, collect each measure from the source named in pilot.json, fill results.json with today's date, run pilot.py score and send the interim line. Do not edit pilot.json, sign off, or publish anything.
```

One prompt.

## Free preview of pilot-scorecard-agree-the-pass-line-before-you-start.md

````markdown
# Pilot Scorecard

**Agree what a passing pilot looks like before it starts. One page of terms the client owner signs, locked pass lines, and a scored report at the end that says pass or fail without an argument.**

Version 1.0, 2026-11-03. By PaperChaseWebb Inc. Free. Works with Grok Bot, Claude Code, Codex and Hermes, or by hand. Python 3.8 or newer, no installs.

## The problem

Most agent pilots end in a debate. The provider points at the hours saved, the client remembers the one bad reply, and nobody wrote down what "worked" meant. Worse, targets drift mid-pilot, so even a good result looks negotiated.

The fix is to write the pass line first: a few numbers, where each comes from, and what happens after a pass or a fail. The client owner signs it. Then the numbers decide.

## What you get out of it

- One terms file: one workflow, 30 days at most, up to 5 measures with today's number, a target and the source, plus data limits, an off switch, the fee and refund rule, and the next step either way.
- A one-page terms sheet the client owner reads and signs at their own terminal.
- Locked pass lines: if the terms change after sign-off, scoring is refused until the client owner signs again.
- A separate, default-no case-study question, so consent is never bundled into the deal.
- A report at the end: met or not met per measure, pass or fail, and the step both sides agreed to in advance.

## Who it is for

Providers and small agencies who offer a trial before a monthly plan. Ow
````

## Test record

- Date: Nov 3, 2026
- What was run: tests/test_pilot.py (28 tests) unpacked from the pack file and run fresh on Linux, Python 3.8, 3.9, 3.11, 3.12 and 3.13; check, sheet, approve and score on the fictional example.
- Result: 28 passed, 0 failed on all five versions; the example passes 2 of 3 measures; sign-off refused in agent sessions, without a terminal, for the wrong name and after a terms edit; scoring refused without sign-off or after the terms change; terms and results are never modified.
- Not run: macOS or Windows. A real client pilot. A live Grok Bot, Claude Code, Codex or Hermes session.

Copied from this item's dated test notes. What was not run stays in.

## Worth knowing

- It scores what you give it. Pick sources both sides can see.
- A short pilot is a small sample. A pass says the workflow met the agreed lines in that window, nothing more.
- On 2026-11-03: 28 tests passed on Python 3.8, 3.9, 3.11, 3.12 and 3.13 on Linux, run from the files unpacked out of the pack. Agent rules guide; they were not loaded inside each harness for this release.
- Not a contract and not legal advice. Your own agreement governs the fee and refund.

## Questions

**Why lock the terms?** So a target cannot move mid-pilot. If the terms change after sign-off, scoring is refused until the client owner signs the new version.

**Can my agent sign off for the client?** No. Sign-off is refused inside an agent session and for anyone but the client owner named in the terms.

**Is case-study consent part of the deal?** No. It is a separate question at sign-off, it defaults to no, and saying no changes nothing else.

**What should I pair it with?** Agent Intake Packet (free) briefs the agent for the pilot without handing over secrets, and Retainer Scope Sheet ($29) itemizes the monthly that follows a pass.

## How to get it

Free sample. Download it directly at https://paperchasewebb.ai/bots/marketplace/items/pilot-scorecard-agree-the-pass-line-before-you-start/, no signup.

Seller: PaperChaseWebb Inc., mgmt@paperchasewebb.com