Everything I Looked At Was Fresh
Three items a week is the entire output of the loop. It reads seven feeds for advances in how people are building with language models, scores everything it pulls, and once a week hands me the top three to go and read properly. Last week’s were three consecutive bullets from the same product changelog. So were the week before’s.
That is not a tie broken badly. It is the only thing the rule can do. The candidate lane held 1,897 stubs in one global ranked order, the selection was .slice(0, 3), and that changelog holds ranks 1 through 8 unbroken and 48 of the top 60. There was never a week where the top three could have come from anywhere else.
The source at the top has never filed anything
Every source carries a track record: how many items it has produced, and how many of those ever became a finding. I keep it so I can retire a source that is not paying for itself. I had not read it in a while. Eight sources are configured, seven still switched on:
pulled filed
company news feed 163,490 0
engineering blog 13,811 2
personal blog 4,440 0
preprint search 3,725 3
product changelog 3,375 0
forum search 2,951 1
changelog 1,264 0
changelog (switched off) 1,070 0
------- -----
194,126 6The changelog that owns the head of the ranking is the fifth row. Three thousand three hundred and seventy-five items pulled, nothing ever filed.
Every finding the loop has produced in its life came from the preprint search, an engineering blog and a forum search. When I measured, those three sources’ best candidates sat at ranks 9, 125 and 241. A batch of three off the top could not reach any of them. It had never reached any of them. The six findings exist because I went and read those feeds by hand, on evenings when I was not using the tool I built to do that.
194,126 items pulled, six filed, none of the six survived vetting. That is the real yield of the whole apparatus, and it is not what this piece is about, but I am not going to quote the interesting number and hide the embarrassing one.
The ranking was not wrong
My first instinct was that the scoring was broken, because a source with a lifetime record of nothing should not hold eight of the top eight ranks. But the scores are defensible one at a time. A changelog entry about tool use really is on topic, really is recent, really does mention the things I told the scorer to care about. It just never turns into anything.
So I did not touch the ranking. The scoring config, the fit config, the surface weights and the lane order came out of this byte-identical. The changelog still sits at rank 1.
What changed is one pass over the ranked list that takes each source’s best unblocked stub before it takes any source’s second. Below the number of distinct sources it falls back to the old order, so a lane holding one source produces the batch it always did.
The live result: rank 1, rank 9, rank 56, where it used to be three rows from the top eight. The batch’s combined fit score went from 111.5 to 94.5. I shipped a selection rule that returns a measurably worse batch by the only number I have for batch quality, which tells you what that number is worth.
What was behind rank nine
The moment the batch reached past rank 8, it started returning rows from sources nothing had visited. I checked how old each source’s best candidate was, expecting to feel good about the fix.
engineering blog rank 125 65.1 days
changelog rank 483 93.4 days
company news feed rank 436 104.2 days
forum search rank 241 106.3 daysThe loop draws a freshness line at 60 days. Past it, a candidate is supposed to drop out of the active lane into the backlog, because a three-month-old post about a fast-moving field is not a research lead any more.
Five of the eight sources had a best candidate already over that line, and the four above are the worst of them. Three by more than three months. These were not obscure rows in the tail. Each one was the freshest thing its source had to offer.
The test ran once, at the door
The rule is a constant and a comparison, and both are fine:
const BACKLOG_AGE_DAYS = 60;
...
return Date.parse(ts) - publishedMs > BACKLOG_AGE_DAYS * 86_400_000;It is called at the instant an item arrives. A stub pulled today is compared against today, sorted into the active lane or the backlog, and never asked again. A candidate that arrived fresh stayed classified as fresh for the rest of its life, whatever the calendar did afterwards.
When I measured, 335 of the 1,897 active candidates were past the line, 17.7 percent, all of them ranking ahead of the entire 1,011-item backlog. That share had been climbing every day since the loop started, by construction, because the only event that could reclassify a stub was an event that happens once and had already happened.
Why it never showed
A changelog for an actively developed product ships something most weeks. Its best candidate is essentially always a day or two old. When I measured, it was 1.4 days.
So every item the system ever put in front of me was fresh, and it was fresh for a reason that had nothing to do with the freshness rule working. I was being shown three rows from the one source in the set that is structurally incapable of going stale. The rule was never enforced on anything I saw, because I only ever saw the input that could not break it.
The monoculture did not just cost me seven sources of input. It cost me the ability to notice anything wrong with them. A ranking, a staleness test, a backlog: I had built all three, and for months I had only ever exercised them against a single well-behaved feed. Concentrating the selection quietly concentrated the testing too, onto the one case that passes.
The one still under the line
The repair re-applies the age test across both lanes on every pull, three times a day. Running it once over the live lanes moved 335 stubs. I checked the sorted set of URLs before and after and the hash is unchanged, so nothing was lost, only relocated.
Each source’s best candidate afterwards:
engineering blog 65.1 -> 7.6 days
forum search 106.3 -> 20.5 days
company news feed 104.2 -> 36.5 days
changelog 93.4 -> 58.3 daysThree of those are genuinely fresh now. The fourth is 58.3 days old, under the line by less than two days, and under it only because there is nothing better behind it. That one is the source I switched off months ago. It has pulled nothing since and cannot, and the repaired selection rule was about to hand me its best surviving post as one of the three things most worth my week.
Next week it crosses the line on its own. The week after, it has no unexpired candidate at all and the batch comes back a row short.
A row short is the right answer. A source with nothing to say should show up as a gap, not as a hundred-day-old post in a slot labelled best available. The old rule never had to produce that gap, because it never asked that source anything.