Pinned Trial Method: test a new skill at one commit, in a proven sandbox
by PaperChaseWebb
Before a new agent skill, plugin or repo touches real work, run it once at a pinned commit inside a sandbox that proves itself first: network off, host read-only, real keys cleared, limits set. Hash the target before and after, scan the output for a planted dummy key, and end with a checked record and a call a person signs.
- Price
- $24
- Difficulty
- Intermediate
- Setup
- 20 min
- Format
- .md file
pinned-trial-method-test-a-new-skill-at-one-commit-in-a-proven-sandbox.md · 33 KB · instant download after checkout
Want this built for you instead? We can build it for you.
What you end up with
A tool trial anyone can repeat: the exact commit, proof the sandbox held, proof the target did not change, and a signed ADOPT, HOLD or REJECT.
What you get
- sandbox_run.sh: one command in bubblewrap with the host read-only, home hidden, network off, env cleared and one writable folder, with CPU, memory and wall-clock limits by profile
- Four probes that run first and stop the trial if the sandbox does not hold
- tree_hash.py: one sha256 for the whole target, so an unchanged target is a fact
- leak_scan.py: finds the planted dummy key anywhere the tool wrote it
- trial_check.py: 21 error checks and 2 warnings on the trial record, read-only
- The seven-step method, three profiles and a blank trial record
- AGENTS.md, CLAUDE.md, a pinned-trial skill, a Grok Bot standing order, Codex and Hermes notes
- 24 tests, including a hostile tool that tries to rewrite the target, write to /etc, reach the network and copy the key
Who it is for
Anyone who wants a hands-on tool trial they can repeat and defend, with a person making the final call.
Works with
Solid chips: the agent is named in this listing's own tools list. Dashed chip: One shell script, four Python scripts, a record template, AGENTS.md and CLAUDE.md rules, a skill file, a Grok Bot standing order and Codex and Hermes notes. The sandbox needs Linux or WSL2.
Tools needed
Cost to run: The kit is the only purchase. bubblewrap and Python are free.
Time to result: First trial in about 30 minutes on the quick profile
How it works
Run the tests
Unpack the kit, install bubblewrap and run python3 -m unittest discover -s tests.
Expected: 24 tests pass, including the hostile tool tests.
Pin and read
Clone the tool from its official address, check out one full commit, confirm it with git rev-parse HEAD and read every instruction file it gives an agent.
Expected: A commit, a license and a list of flagged instructions in the record.
Hash, then run in the sandbox
Hash the target with tree_hash.py, then run the tool through sandbox_run.sh on the quick profile.
Expected: probes.json with four true values and the tool's output in the run folder.
Check and sign
Hash the target again, run leak_scan.py, fill the record and run trial_check.py. A named person signs the call.
Expected: CLEAN from the checker and a signed verdict.
Sample prompts, free to use
Follow AGENTS.md in this kit. Trial <tool> from <official URL> at commit <hash> against <target folder> on the quick profile. Stop after trial_check.py and give me its output, your proposed call, and what you did not test.
Prompt 1 of 2.
Read trial.json and the files in its evidence list. Tell me whether the evidence supports the call, and list anything the record claims that the files do not show.
Prompt 2 of 2.
Free preview of the file
# Pinned Trial Method **Try a new skill or repo once, at one exact commit, in a sandbox you have proven, against a folder it cannot change. End with a verdict you can defend and a person's signature on it.** Version 1.0, 2026-11-03. By PaperChaseWebb Inc. Works with Grok Bot, Claude Code, Codex and Hermes, or on its own. Linux or WSL2 for the sandbox; the checkers run anywhere with Python 3.8 or newer. ## The problem Reading a tool's README tells you what it claims. It does not tell you what it does when it runs. Most "we tried it" notes leave out the version, where it ran, whether it touched anything, and who decided. Two weeks later nobody can repeat the test or explain the yes. ## What you end with One trial record per tool, checked by a script, with four facts nobody has to take on trust: 1. **The exact code.** A full 40-character commit, confirmed after checkout, plus its license. 2. **A proven sandbox.** Four probes pass before the tool runs: network blocked, host read-only, environment cleared of real keys, limits set. 3. **An untouched target.** One sha256 of the whole target folder before and after. They match or the trial says so. 4. **A signed call.** ADOPT, ADOPT WITH LIMITS, HOLD or REJECT, with evidence files, what was not tested, and a named person who approved. The agent never signs. ## Who it is for - Teams adding agent skills and MCP servers faster than anyone can read them - Agencies that install tools for clients and need a record of what was test
Test record
- Date
- Nov 3, 2026
- What was run
- tests/test_trial_check.py and tests/test_desk.py (24 tests) on Linux with bubblewrap, Python 3.8, 3.9, 3.11, 3.12 and 3.13, including a hostile tool run inside the sandbox; the free sample unpacked from its file and run fresh.
- Result
- 24 passed, 0 failed on all five versions; each of the 21 record faults caught by its own code; the hostile tool changed nothing and its copied key was found.
- Not run
- macOS or Windows. WSL2. A live Grok Bot, Claude Code, Codex or Hermes session.
Copied from this item's dated test notes. What was not run stays in.
Worth knowing
- The sandbox guards against honest mistakes and overreaching instructions. It is not a malware lab; do not trial code you already suspect is hostile on a machine that matters.
- Tools that need the network to work at all cannot be judged in this sandbox. Record them as HOLD and plan a separate, approved test.
- The sandbox script needs Linux or WSL2 with bubblewrap. The Python checkers run anywhere.
- The memory limit counts virtual memory, so some runtimes need the standard profile.
- On 2026-11-03: 24 tests passed on Python 3.8, 3.9, 3.11, 3.12 and 3.13 on Linux with bubblewrap. Agent rules guide; they are not a lock.
Questions
How is this different from Vet Before You Install?
Vet Before You Install is the read-only review you do first. This is the hands-on run after it, with proof the sandbox held and the target did not change.
Can my agent run the trial?
Yes. It can pin, read, run the sandbox and fill the record. It cannot sign the adoption; the checker fails a record where the runner approved itself.
Does it work on a Mac?
The checkers do. The sandbox script needs Linux, so on a Mac use a Linux VM or a cloud box.
What does a trial cost?
Nothing beyond the kit. The record makes you write the money and minutes the run took, and zero is a valid answer.
Details
Version 1.0 · Updated Nov 3, 2026 · Markdown version of this page
License: Use for yourself, your team or your clients. Do not resell or republish the kit itself. Download terms
Related
Pinned Trial Method: the trial record checker (free sample)
The read-only checker and a clean example record from Pinned Trial Method. Fill in a record after any hands-on test of a new skill or repo and see what it is missing: the pinned commit, sandbox proof, an unchanged target, a leak scan or a person's sign-off.
Vet Before You Install: twelve checks, a sandbox recipe and a tested vetting folder for third-party agent code
Decide whether a skill, plugin, mod, MCP server, connector, repo or model gets into your agent setup: sort the seat, read every file that runs, map reach and the data path, pin and vendor it, trial it in a sandbox with a planted fake key, then record a decision the owner approves at their own terminal. v1.1 adds a seat sort before any vetting, two new checks (data path and reach, approval surfaces), evidence tiers, twelve token classes including hidden text, encoded strings, startup file edits and approval hooks, a fresh-session second read, data path questions for proxies and routers, dependency and model weight pinning, a fifth decision (not vettable) with recheck dates and an approved shelf, more special cases, a vetting folder with a fifteen-check read-only checker and an owner approval helper, agent files for Grok Bot, Claude Code, Codex and Hermes, and twelve new prompts (27 in all). Price unchanged.
Replicate Before You React: a trial kit for claims about AI agents
Test a claim about a coding agent on your own setup before you change anything: a security hole, a token-saving prompt, a hidden cost, a skill that makes the agent better. Restate the claim, run each setup 10 to 20 times in fresh containers with one capped key, use a harmless canary instead of a real payload, run a control arm for cost and quality, and get rates with honest intervals and a result card that says what you did not test.
Extension Lockfile: pin your editor extensions and check for persistence after a bad update
Write down the editor extensions you trust at the versions you reviewed, turn off silent updates, and check every machine against that list with a read-only script. Take a baseline of SSH keys, startup entries, crontab and shell files on a quiet day, so when the next bad extension update is announced you can see what changed in one command. Includes an indicator file and runbook for the 2026-10-29 Claude Code extension incident.
Template Safety Card: what it can touch, what needs your yes, what it was tested on
One honest page for every agent template you sell or share: what it can touch, what waits for the buyer's yes, what it never does, which secrets it needs and what it was really tested on. A checker fails the template when the card is missing or does not match the files: a network call the card never mentions, texts sent with no yes, a key name left off, 25 prompts claimed for 3, a test row with no saved log, a card from an older version. It checks your listing against the card too. You approve each card at your own terminal.