Free during early access.
Every month we plant 16 realistic bugs in your code, one at a time, and run your own tests against each. During early access that is free: the monthly run, every miss with the test that would catch it, and test pull requests for the misses you want. Public or private repos.
What you get every month
- 16 realistic bugs, planted one at a time. Chosen from failures seen in real AI-written code, written without reading your tests, and frozen before anything runs.
- Your score, and its trend. Measured on the same bugs month to month, so the trend is real.
- Every miss, with the test to add. The bug, the file and line, whether your tests never run that code or run it without checking it, and the test that would have caught it.
- Held. All of last month's bugs are planted again. Misses that now get caught prove the tests you added work; catches that now slip show a test was deleted or weakened.
- New. Up to 16 new bugs each month, in whatever changed since the last run first, each miss with the test to add.
- Test pull requests, if you want them. For each miss you want closed, we open a pull request that adds the test. Only with your OK.
- A score you can show. When you want to, we switch on a public receipt page and a badge for your README. They show the score, the date and how it was measured, never the misses. Off until you ask; one reply switches it off again.
The first month
We copy your repo into a private workspace, get your test suite running exactly as your CI runs it, and check it passes on your unmodified code several times so flaky tests can't pose as catches. Then we write the 16 bugs, send you a hash of the bug list before anything runs, and deliver the full report, usually within a week of access.
Real bugs come first
If a run ever turns up a genuine bug in your shipped code, we tell you separately, whether or not you go further. We take a few new repos each week, in the order they arrive.
What we need from you
- Read access to one repo. Nothing is ever pushed to it. Public repos need nothing.
- A test suite that runs in CI. If it needs a database or another service, we start it in a container the way your CI does. It never needs your production secrets.
- Size and stack. Repos up to about 100,000 lines of TypeScript, JavaScript or Python; Go and Java on request. Larger repos, or several, also on request.
Questions
What counts as a caught bug? A test that passes on your unmodified code fails with the bug planted, and fails again on a second run. Anything else is a miss or, if the bug broke the build, not counted.
Can we see the bugs before you run them? You get a hash of the bug list first, so it can't change after the fact. The list itself arrives with the report.
Where does our code go? A private copy on our CI, deleted within 30 days after you stop. Security and data has the detail.
Why every month? Because the suite you measured in March is not the suite you have in June. Code changes, tests get deleted or loosened to make a build pass, and an assistant rewriting a module rarely rewrites its tests. Each month all of last month's bugs are planted again, so the tests you added are proven and a weakened test shows up as a miss, and up to 16 new bugs go into whatever changed since the last run. One run tells you where you stand. The series tells you whether it's holding.
Will it stay free? It is free during early access. If we ever introduce paid plans, we tell you by email at least 30 days before, and nothing is charged without your agreement. The terms say the same.
How do we stop? Reply to any report, or email sk@trenyx.io. Your next month doesn't run, and the private copy is deleted within 30 days.