A Banned Word Is Not an Outage
Written later and filed to this date, as part of filling in every day of the year.
The subject line reads like a ruling: fix(services): a streaming content block is a provider success, not a breaker failure. It landed on a Thursday morning in June, eight minutes and forty-four seconds after the commit it corrects, from the same session, in a Go service that stands in front of several language models and promises to switch between them when one goes down.
The streaming path it fixes had been written minutes earlier. A provider emits an answer in chunks; a filter watches the chunks for banned words and, in block mode, stops the stream before the offending piece ships; the circuit breaker counts each provider’s consecutive failures and, past a threshold, skips that provider for a cooldown. The first cut already knew that a block must not fail over. Its own comment said so: a deterministic refusal, and a different provider would not change the policy. What it got wrong was the ledger. Any error at the end of the stream, including the filter’s refusal, was recorded as a provider failure before the control flow decided what to do about it.
The body names the consequence precisely: recording it that way “would trip the circuit breaker on a healthy provider that merely emitted a banned word.” Then it draws the line the fix is built on. An error that comes back through the emit callback, the filter refusing the output or the client’s connection dropping mid-write, is downstream of the provider. The provider did its job. Only an error from the provider’s own stream is a provider failure, and only that kind is eligible for failover, and only before the first byte has gone out. The buffered path, the body notes, had always behaved this way by accident of ordering: its filter runs after a successful generate, so a block there never touched the breaker.
What the author understood is that a failure belongs to the layer that failed, and that a refusal is a policy working, not a dependency breaking. The test it added says it in its name, ContentBlockIsNotProviderFailure: a provider streams “clean text” and then a forbidden word, the caller gets the block error, and the provider’s failure count reads zero and its success count reads one.
What the commit does not say is what the first cut would have done to the service as a whole. Twenty-seven minutes earlier another session had taught the readiness probe to answer not-ready when every provider’s circuit is open. Chain the two: enough callers asking for answers with banned words in them, and every healthy provider’s circuit opens, and the service reports itself down. A load balancer would have pulled it for refusing to say things it was built to refuse to say. The fix closes that path without mentioning it.
One more thing the body leaves out, because the repository says it elsewhere. The breaker’s threshold defaults to zero, which means the breaker does not exist until someone configures it, and the only providers that have ever stood behind it are stubs that echo the prompt back behind a fixed prefix, split on spaces for the streaming path. In every environment this service has run in, the banned word a provider would have been punished for emitting was a word the caller typed.