Privacy

Accept optional first-party analytics or decline. Functional journey and sound preferences stay on this device.

Read the privacy notice

Open to talks and workshops

All articles

Part of The AI Failure Files · 1/2

AI wrote the code. I reviewed it. Three bugs still shipped.

Three bugs that shipped in AI-assisted code, and the gates that catch them now

A hook after an early return, a silent embedding wipe, a reranker that vanished with a dead key. None of them were caught by reading the code.

I review every change the model produces. These three still got through, and each one changed what "review" means on this site.

1. A hook after an early return. A React component returned early on one branch, and a hook sat below that return. On mobile the assistant mounted twice. The code read fine; the bug is structural, not visible. A stricter lint configuration caught it the day it was turned on. Lesson: the review that finds this is a tool, not a person, so the tool has to exist.

2. A silent embedding wipe. Chunks that exceeded the embedding model's limit failed, and the sync treated the failure as "nothing to write", wiping whole batches of vectors. Parts of the site quietly vanished from the assistant's memory with no error anywhere. Found during a retrieval audit, and now covered by retrieval evaluation cases that run after every reindex, so the same silence would fail a check instead of passing unnoticed.

3. A reranker that vanished. When a provider key expired, the reranking step stopped running and the pipeline carried on with weaker ranking, reporting nothing. Fixed with provider health checks and a circuit breaker, so the system now says it is degraded instead of pretending it isn't.

What they have in common. All three failed quietly. None were wrong in a way a diff shows. The gates that exist now, lint, 169 tests on Alex OS, a fixed 30-question evaluation set, and per-call tracing, are not there because I distrust the model. They are there because I distrust silence.

Published Last updated