The Parts Were Right and the Whole Was Wrong
NewsAgg used to rank articles with one number: a local model’s holistic opinion of how valuable a piece was. That’s unauditable by design (you get a float, not a reason), so I replaced it with something that sounded like the responsible version of the same idea: a portfolio of roughly a hundred small, independent signals (does the piece cite its sources, does it link the paper it’s describing, is the writing hedged or hyped, and so on), fused into seven named facets, each signal individually unit-tested against hostile input. The pitch, in the commit that shipped it, was that the algorithm should be common sense: separate fact from non-fact, then rate the writing on its own merits, and the good stuff floats up on its own.
Every piece of that was correct. Then I ran it against the real corpus and read what came out the other end.
Merit is not relevance
The top of an AI-research feed read: “Summer Games Done Quick raises $2 million,” followed by an article about weightlifting and blood sugar. Both are honest, well-attributed, carefully sourced pieces of writing. The engine scored them exactly right: high evidence, high factuality, no hype. That’s the problem. I’d built a merit detector and pointed it at the whole internet’s worth of merit, most of which has nothing to do with why anyone opens this particular feed. A brilliant article about nothing you came here for isn’t a good result, and neither is an important story reported badly; the rank needed both terms, not either one. The fix was a geometric mean of merit and relevance, so a zero on either side can’t be bought back by an excess of the other. Simple in hindsight. Invisible until I actually looked at the top ten.
A correctly enforced rule can still lose
The digest caps how many arXiv preprints can appear per issue, so it doesn’t turn into a wall of papers. The cap worked exactly as written, and the digest still came back as three papers followed by a Pokémon Go event recap and a listicle about AWS Lambda. The bug wasn’t the cap; it was the pool the cap drew from. If the candidate pool feeding a top-ten selection is 95% preprints, capping preprints at three doesn’t surface the other seven good items; it just backfills with whatever’s left after the cap runs out of real candidates, which is filler. The good outlets were genuinely in there, just buried under two hundred and fifty papers a day. Widening the pool before applying the cap, not changing the cap, got eight distinct outlets into one digest instead of ten papers or three papers and scraps.
A true fact, wrongly applied
The engine also tracks financial and organizational ties, so a reader can see who has a stake in a story being told a certain way. Early on it flagged “Alphabet owns Google DeepMind” as a conflict of interest on every single DeepMind post. That’s a true fact, and applying it as a conflict flag on every post from that org is wrong, because ownership is identity, not an outside interest. Y Combinator owning Hacker News isn’t a conflict on an article about sound cards; it’s just who the publisher is, background true of everything they publish. The distinction that survived: an author or outlet’s founding, financial, or advisory tie to a company the piece is about is worth an alarm; the org chart the piece was published under is not. Get that wrong in the panicky direction and the panel cries wolf on every post from every lab, and the one story that actually deserved the flag (a Google Cloud deal announced by the company Google partly owns) goes out looking exactly as loud as everything else.
The shape underneath all three
None of these were logic errors. The merit score was computed correctly. The cap was enforced correctly. The ownership fact was true. Every part, read on its own, passed review. What failed was the composition (the step where correct pieces combine into a ranking, a digest, an alarm), and that failure mode doesn’t show up by reading code or by watching unit tests go green, because each unit really was fine. It only shows up when you make yourself read the actual output an ordinary reader would see, and it looks wrong the instant you do: a charity fundraiser leading an AI feed, ten preprints where a digest should be, a conflict flag on a company being exactly what it obviously is.
The structural fix, once I’d found the pattern, was to stop trusting the score and force the system to show its work. Every ranked item on NewsAgg now carries a plain-English receipt: the specific signals that raised it, the ones that held it back, and, when the engine genuinely doesn’t know, an honest admission of that instead of a confident guess dressed up as a number. That’s not a cosmetic feature. It’s the thing that would have caught all three of today’s bugs without me having to stumble onto the SGDQ headline first: if the system can’t explain a rank in a sentence a reader would accept, the rank is wrong, whether or not the code that produced it passed its tests.
The individually-correct components were never the risk. The risk was believing that individually correct adds up to collectively correct without checking, and the only check that actually works is reading what the thing you built really says, out loud, as a stranger would read it.