Tests projects have merged
Tests from our planted-bug runs that open-source maintainers have taken into their own projects.
We plant a small, realistic bug in a private copy of a project and run its own tests. When a bug gets through, we write the test that would have caught it. Before we offer that test as a pull request, we check that it passes on the project's current code and fails with the bug in place.
None of these were live bugs. The planted bug only ever existed in our private copy, and each pull request adds tests without changing product code. The tests are written with Claude Code, and SK approves every pull request before it opens. This page lists only tests the maintainers accepted, with a link to each one. Newest first.
-
moov-io/ach: the $25,000 limit on check-conversion payments
ARC entries are capped at $25,000, counted in cents. The tests covered an amount over the cap, but the sample entry was only $250, so a cap mistakenly written in dollars would have rejected every real payment over $250 with all tests still green. The new test checks that exactly $25,000.00 is accepted.
-
pandera: signed and unsigned integer types
Pandera's integer check compares whether a type is signed as well as its size, but no test relied on that. Without it, a uint8 column would accept negative int8 data. The new test checks each signed and unsigned pair of the same size is rejected, then accepted once the data has the right type.
-
gosec: the severity of its most serious taint rules
Two rules are declared critical, and gosec reports them as high. Nothing ran a finding through that mapping and checked the result, so a slip there would quietly drop those rules out of high-severity scans. The new test checks each severity level.
-
mcp-go: a resumed stream that gets its answer live
Every resume test let the request finish first, so the final response always came from the replay. Nothing checked a client that reconnects while the request is still running. If the stream stayed open after that live answer, the client would wait forever. The new test resumes mid-request and checks that the stream ends.
-
slack-go: the time window sent when reading channel history
Nothing checked that a caller's latest and oldest timestamps reached Slack in the right fields. A mix-up would still succeed, but return the wrong span of messages. The new test sends both and checks each one.
-
Okteto: port forwards built from a compose file
Every existing case used matching ports such as 8080:8080, so swapped host and container ports would have gone unnoticed. The new test pins 8080:80 to local port 8080 and remote port 80.
-
SurfSense: which prompt a 7B local model gets
The project's docs give a 7B model the capable prompt, but no test sat on that boundary, and 7B is one of the most common local sizes. The new test holds a 7B model to the capable prompt.
-
Dolt: spotting a changed foreign key during a merge
Merges use EqualDefs to tell whether a branch changed a foreign key, but no test checked that a change to only ON DELETE or ON UPDATE counts as a change. The new test does.
-
alpaca-py: retrying after a rate limit
The client says it retries on a 429 response, but no test exercised that path. The new test serves a 429 and then a 200, and checks that the call retries and succeeds.
-
CrewAI: the skill catalog and checkpoint restore
One test checks that a skill already activated isn't listed as still loadable. The other checks that after a checkpoint restore, a task keeps its own expected output instead of its description.
-
Pingora (Cloudflare): seven cache and HTTP behaviours
Seven unit tests covering cache revalidation, cache variants, serve-stale windows, the cache lock, and HTTP targets and headers. Cloudflare's maintainers synced them to main themselves, so GitHub shows the pull request as closed rather than merged.
How a run works is on the planted bugs page. If you maintain one of these projects and would rather not be listed, email sk@trenyx.io and we'll take it off.