Replicate a scary agent security claim with a harmless canary before you react
by PaperChaseWebb
After a security firm reported that a planted AGENTS.md in a Downloads folder could hijack coding agents, one engineer rebuilt the test with a harmless canary in throwaway Docker containers: Codex and Claude Code, default and sandboxed, 20 runs each, and saw 0 hijacks out of 80.
- Price
- Free
- Difficulty
- Intermediate
- Setup
- 30 min
- Format
- .md file
replicate-a-scary-agent-security-claim-with-a-harmless-canary-before-you-react.md · 3 KB · direct download, no signup
What you end up with
The result is not that the attack is fake; it is that it did not reproduce on those versions and settings, and the original may have used other versions, folders or prompts.
What you get
- A plain-language summary of the workflow
- A fill-in prompt template
- What to watch out for
- A link to the original source
Who it is for
Builders who already use AI agents and want a proven workflow they can copy into their own project.
Works with
Solid chips: the agent is named in this listing's own tools list. Dashed chip: Plain Markdown, so it also pastes into other agents such as Grok Bot, Hermes, OpenClaw or Muse. We list those as compatible by format only, not as tested.
Tools needed
Cost to run: Free with the agent plan you already use.
Time to result: One working session
How it works
Write down the exact claim: which agent, version, setting and success
Write down the exact claim: which agent, version, setting and success rate.
Expected: Done before you move to the next step.
Replace any harmful payload with a canary, such as creating an empty m
Replace any harmful payload with a canary, such as creating an empty marker file.
Expected: Done before you move to the next step.
Build a matrix of agents and modes and run each cell at least 20 times
Build a matrix of agents and modes and run each cell at least 20 times in fresh throwaway containers.
Expected: Done before you move to the next step.
Count canary hits per cell and record versions and settings
Count canary hits per cell and record versions and settings.
Expected: Done before you move to the next step.
Report the result with its limits, including what you did not test
Report the result with its limits, including what you did not test.
Expected: Done before you move to the next step.
Sample prompt from the pack
Apply this workflow to [your project]: 1. Write down the exact claim: which agent, version, setting and success rate. 2. Replace any harmful payload with a canary, such as creating an empty marker file. 3. Build a matrix of agents and modes and run each cell at least 20 times in fresh throwaway containers. 4. Count canary hits per cell and record versions and settings. 5. Report the result with its limits, including what you did not test. Stop and ask me before anything irreversible: deploys, payments, sends or deletes.
Adapted by PaperChaseWebb. Not the original author's prompt.
Free preview of the file
# Replicate a scary agent security claim with a harmless canary before you react PaperChase agent use case (free feed entry), version 1.0. Source: [Chris Johnson (@ComputeGhost)](https://x.com/ComputeGhost/status/2115299194454028514), 2026-10-28. ## What they did The original post claimed high hijack rates when an agent is launched in a folder that contains a planted instruction file. He replaced the payload with a canary (an instruction to write a marker file that does nothing), ran each agent in its default mode and in its sandboxed mode, 20 runs per cell, in fresh containers, and counted how often the canary appeared. It never did on the versions he tested. ## Worth knowing The result is not that the attack is fake; it is that it did not reproduce on those versions and settings, and the original may have used other versions, folders or prompts. The method is the takeaway: a canary instead of a real payload, a matrix of agents and modes, enough runs per cell, fresh containers, and a report that names versions. It is a good habit before you change your setup because of one alarming post.
Worth knowing
- The result is not that the attack is fake; it is that it did not reproduce on those versions and settings, and the original may have used other versions, folders or prompts. The method is the takeaway: a canary instead of a real payload, a matrix of agents and modes, enough runs per cell, fresh containers, and a report that names versions. It is a good habit before you change your setup because of one alarming post.
- Figures are as reported by the original author and were not re-measured by us.
Questions
Where does this come from?
A public post credited to Chris Johnson (@ComputeGhost). Source: https://x.com/ComputeGhost/status/2115299194454028514
Is it free?
Yes. Feed entries are free to read and download.
Details
Version 1.0 · Updated Oct 29, 2026 · Markdown version of this page
License: Personal and commercial use. Credit the original source when you share the workflow. Download terms
Related
AGENTS.md, CLAUDE.md and Cursor rules starter files
Six fill-in instruction files for coding agents (base, Next.js plus Supabase, FastAPI, browser game, CLAUDE.md pointer, scoped Cursor rules), an Approvals section that makes the agent stop and wait for your yes, a table of what Grok Bot, Claude Code, Codex, Hermes and Cursor each read, a read-only checker for your rule files and 9 prompts to write, trim and test your own.
Ask-first rules for AI agents: hard nos, approval cards and a test set
Paste-in rules that make an agent ask before it sends, spends, posts or deletes, define what a real yes is, and stop retries after a no, plus 12 test scenarios to check it.