A pre-registered, blind verification pass on jakethehoffer/ledgerly, a TypeScript engine that turns Stripe webhook events into balanced double-entry journal entries. Free and unsolicited. Verdict: exemplary.
Most test suites are green. The question that matters isn't whether the suite passes. It's whether the suite would catch a money bug introduced tomorrow. So I ran ledgerly's own 810 tests as shipped (all green), then injected sixteen semantic money-defects one at a time and measured how many its suite caught.
| result | |
|---|---|
| baseline suite (as shipped) | 810 / 810 green |
| planted money-bugs caught | 14 / 16 |
| confirmed defects in shipped code | 0 |
| findings disclosed | 1 (low, benign) |
Fourteen of sixteen is the best result of any target I've put through this, and more tellingly, its suite caught the balanced-but-wrong mutations: entries that still balance to the cent (debits = credits) but post to the wrong account. Those are the ones weaker suites wave straight through, because a balance check can't see them. ledgerly's tests assert the specific accounts, not just the arithmetic.
The two defects that escaped both trace to a single localized gap, in the tests, not the shipped code. One reconciler fixture re-spreads its amount over months that divide evenly (70000 over 7 = 10000 each, no remainder), so the branch that places a leftover cent on the last month is never exercised. A regression there would ship green. The fix is one fixture whose numbers don't divide evenly. I filed it, gratefully, as a public feature request: ledgerly#2. There is no security or money bug in the shipped code.
Before reading a single line of ledgerly's implementation, I wrote the attack plan from its README alone, hashed it, and posted the hash publicly. That public post is an external clock: it proves the plan was fixed while I was still blind, so nothing in the findings could have been reverse-engineered after the fact. You can check the whole chain in about a minute:
shasum -a 256 00-preregistration/ATTACK-PLAN.mdEvery engagement ships the pre-registration, the baseline, the planted-defect matrix, and the findings, disclosed to you first, privately, with the same receipts you see here.