# Judge agent-written libraries with test suites your side did not write

> A team built open-source C# libraries with a factory of agents (planner, project manager, orchestrator, engineers, reviewers) and scored each one against public conformance suites such as test262 and html5lib, publishing the failures too.

- URL: https://paperchasewebb.ai/bots/marketplace/items/judge-agent-written-libraries-with-test-suites-your-side-did-not-write/
- Type: Agent use case pack
- Category: Vibe coding
- By: PaperChaseWebb
- Works with: Claude (named in the listing); Plain .md
- Price: Free USD
- Difficulty: Advanced
- Setup time: 1 hr
- Time to result: One working session
- Format: Markdown (.md) file, judge-agent-written-libraries-with-test-suites-your-side-did-not-write.md
- Version: 1.0, updated 2026-10-27
- License: Personal and commercial use. Credit the original source when you share the workflow.

## What you end up with

When an agent writes both the code and the tests, passing tests prove little.

## What you get

- A plain-language summary of the workflow
- A fill-in prompt template
- What to watch out for
- A link to the original source

## Who it is for

Builders who already use AI agents and want a proven workflow they can copy into their own project.

## Tools needed

- Claude Opus and Sonnet: specialist agents in the factory
- GPT models: specialist agents in the factory
- Public conformance suites: outside judge of correctness
- You need: An AI coding agent or chat model
- You need: A small real task to try it on

Cost to run: Free with the agent plan you already use.

## How it works

1. **Find the official or most widely used conformance suite for what you a.** Find the official or most widely used conformance suite for what you are building Expected: Done before you move to the next step.
2. **Wire it in as the only definition of done.** Wire it in as the only definition of done Expected: Done before you move to the next step.
3. **Let agents read failures, never edit the suite.** Let agents read failures, never edit the suite Expected: Done before you move to the next step.
4. **Publish the pass rate and the failing cases.** Publish the pass rate and the failing cases Expected: Done before you move to the next step.
5. **After each release, fix the process before starting the next build.** After each release, fix the process before starting the next build Expected: Done before you move to the next step.

## Sample prompt: Workflow template (Claude Opus and Sonnet, GPT models)

```
Apply this workflow to [your project]:
1. Find the official or most widely used conformance suite for what you are building
2. Wire it in as the only definition of done
3. Let agents read failures, never edit the suite
4. Publish the pass rate and the failing cases
5. After each release, fix the process before starting the next build
Stop and ask me before anything irreversible: deploys, payments, sends or deletes.
```

Adapted by PaperChaseWebb. Not the original author's prompt.

## Free preview of judge-agent-written-libraries-with-test-suites-your-side-did-not-write.md

````markdown
# Judge agent-written libraries with test suites your side did not write

PaperChase agent use case (free feed entry), version 1.0. Source: [u/ai-tacocat-ia on r/vibecoding; Kurtis Welch, Recursive AI](https://www.reddit.com/r/vibecoding/comments/1xgvsu1/ai_code_is_slop_has_a_problem_test262/), 2026-10-27.

## What they did

Recursive AI describes a factory that rewrites an existing kind of library from scratch in a target language, with no dependencies beyond the base library. Specialist agents plan, manage, build and review, following a process written into their shared prompts. Each library is judged by an outside test suite rather than tests the agents wrote: the post cites 99.55% on test262 for a JavaScript engine, 100% on html5lib for an HTML parser, 330 of 330 on a shader compiler suite and byte-identical output on the Opus audio test vectors. After each 1.0 release the owner adjusted the factory before the next run, and owner messages per library fell over time.

## Worth knowing

When an agent writes both the code and the tests, passing tests prove little. A suite written by someone else, ideally the standard's own, is a fair judge, and publishing the misses keeps the claim honest. All scores are the team's own; this is a vendor research write-up.
````

## Worth knowing

- When an agent writes both the code and the tests, passing tests prove little. A suite written by someone else, ideally the standard's own, is a fair judge, and publishing the misses keeps the claim honest. All scores are the team's own; this is a vendor research write-up.
- Figures are as reported by the original author and were not re-measured by us.

## Questions

**Where does this come from?** A public post credited to u/ai-tacocat-ia on r/vibecoding; Kurtis Welch, Recursive AI. Source: https://www.reddit.com/r/vibecoding/comments/1xgvsu1/ai_code_is_slop_has_a_problem_test262/

**Is it free?** Yes. Feed entries are free to read and download.

## How to get it

Free sample. Download it directly at https://paperchasewebb.ai/bots/marketplace/items/judge-agent-written-libraries-with-test-suites-your-side-did-not-write/, no signup.

Seller: PaperChaseWebb Inc., mgmt@paperchasewebb.com