TermOctober 10, 2026
Circuit breaker (for LLM calls)
- Category
- AI & LLMs
- Related terms
- Fallback
- Graceful degradation
- Provider health
01
One-line definition
A switch that stops calling a failing model provider for a while, so requests fail fast and fall back instead of waiting on retries.
02
Plain-English explanation
After certain errors the breaker "opens" and callers skip that provider. After a cooldown it lets one request through to test it. For LLM providers, credit and authentication errors should open it at once, while a plain 400 never should: that's a bug in your request, not an outage.
03
Real example from my work
Before I added one, every question on my site paid about three failed model calls with retries when the API credit ran out. Afterwards the fallback answered in under a second.
04
Why it matters
Without it, a dead key makes a feature slow as well as worse.