# Judge agent-written libraries with test suites your side did not write

PaperChase agent use case (free feed entry), version 1.0. Source: [u/ai-tacocat-ia on r/vibecoding; Kurtis Welch, Recursive AI](https://www.reddit.com/r/vibecoding/comments/1xgvsu1/ai_code_is_slop_has_a_problem_test262/), 2026-10-27.

## What they did

Recursive AI describes a factory that rewrites an existing kind of library from scratch in a target language, with no dependencies beyond the base library. Specialist agents plan, manage, build and review, following a process written into their shared prompts. Each library is judged by an outside test suite rather than tests the agents wrote: the post cites 99.55% on test262 for a JavaScript engine, 100% on html5lib for an HTML parser, 330 of 330 on a shader compiler suite and byte-identical output on the Opus audio test vectors. After each 1.0 release the owner adjusted the factory before the next run, and owner messages per library fell over time.

## Worth knowing

When an agent writes both the code and the tests, passing tests prove little. A suite written by someone else, ideally the standard's own, is a fair judge, and publishing the misses keeps the claim honest. All scores are the team's own; this is a vendor research write-up.

## Workflow template

```text
Apply this workflow to [your project]:
1. Find the official or most widely used conformance suite for what you are building
2. Wire it in as the only definition of done
3. Let agents read failures, never edit the suite
4. Publish the pass rate and the failing cases
5. After each release, fix the process before starting the next build
Stop and ask me before anything irreversible: deploys, payments, sends or deletes.
```

## Steps

1. **Find the official or most widely used conformance suite for what you a.** Find the official or most widely used conformance suite for what you are building
2. **Wire it in as the only definition of done.** Wire it in as the only definition of done
3. **Let agents read failures, never edit the suite.** Let agents read failures, never edit the suite
4. **Publish the pass rate and the failing cases.** Publish the pass rate and the failing cases
5. **After each release, fix the process before starting the next build.** After each release, fix the process before starting the next build

## Credit

Workflow reported by u/ai-tacocat-ia on r/vibecoding; Kurtis Welch, Recursive AI: https://www.reddit.com/r/vibecoding/comments/1xgvsu1/ai_code_is_slop_has_a_problem_test262/
Related: https://recursiveai.org/research/alera-library-factory/
Related: https://github.com/orgs/recursive-ai-oss/repositories

Summary and template written by PaperChaseWebb in our own words. Questions: mgmt@paperchasewebb.com