Three bugs that shipped in AI-assisted code, and the gates that catch them now
A hook after an early return, a silent embedding wipe, a reranker that vanished with a dead key. None of them were caught by reading the code.
Key takeawayReading AI-written code is necessary and not sufficient. Lint, tests, evaluation cases and health checks are the review that scales.
Overview
I review every change the model produces. These three still got through, and each one changed what "review" means on this site.
1. A hook after an early return. A React component returned early on one branch, and a hook sat below that return. On mobile the assistant mounted twice. The code read fine; the bug is structural, not visible. A stricter lint configuration caught it the day it was turned on. Lesson: the review that finds this is a tool, not a person, so the tool has to exist.
2. A silent embedding wipe. Chunks that exceeded the embedding model's limit failed, and the sync treated the failure as "nothing to write", wiping whole batches of vectors. Parts of the site quietly vanished from the assistant's memory with no error anywhere. Found during a retrieval audit, and now covered by retrieval evaluation cases that run after every reindex, so the same silence would fail a check instead of passing unnoticed.
3. A reranker that vanished. When a provider key expired, the reranking step stopped running and the pipeline carried on with weaker ranking, reporting nothing. Fixed with provider health checks and a circuit breaker, so the system now says it is degraded instead of pretending it isn't.
What they have in common. All three failed quietly. None were wrong in a way a diff shows. The gates that exist now, lint, 165 tests on Alex OS, a fixed 27-question evaluation set, and per-call tracing, are not there because I distrust the model. They are there because I distrust silence.