A Link Is Not a Vote
Two of the ten papers on the brief’s front page carried the same receipt: Links directly to the primary artifact (github.com). That line is worth a full point on the signal that rewards a piece for pointing at the thing it describes. One of the two papers had 314 links in it. Two of them had a host at all, and the only link to github was in the reference list.
Where the links were
My news aggregator folds around a hundred signals into a merit score, and the heaviest facets reward evidence: a paper that links its code, cites primary sources, shows its numbers. In early September it started reading arXiv papers in full instead of stopping at the abstract, and a full paper arrives with everything attached: the body, then References, then the appendix. None of the link-reading signals knew the difference.
A replay over fourteen days put numbers on what that bought. Of every link in the window, 93.6 percent were intra-document anchors, the #bib.bib4 and #S2.SS0.p1 that arXiv’s HTML uses for each inline citation. The classifier required a host before it would call a link “self”, so a link with no host counted as a link out, and 741 papers were credited with citing outside sources, with receipts naming as many as 129 that they did not have. Of the links that genuinely reached a primary record, 77.6 percent sat in the supporting matter: of 2,774 DOI links, 182 were in a paper’s own argument. 838 articles were credited with linking code. 368 of them shipped none, 162 purely on the strength of a reference-section link, and 736 papers earned the “paper” bucket for the act of citing other papers.
Then the prose signals. Seven of them measure a piece’s own writing: claim density, how many of its numbers come with context, the tics a language model leaves behind. They had been scoring the bibliography as the author’s prose. Where a paper had those sections, they were a median 26.5 percent of its plain text, and 55 percent at the ninetieth percentile: names and years at extreme density, nobody’s sentences but the people being cited.
The fix was the same cut three times, applied to three different readers over two days. References, Bibliography and Appendix come off before any link or sentence is read, and a link that lives only there is a citation: scored below a body link to the primary record, never above it. The engine’s version went from seven to eight to nine so the window was rescored under each rule. On arXiv the primary-source signal’s mean fell from 0.60 to 0.42.
Where the sources came from
Four days later the same mistake surfaced at the other end of the pipeline, where the aggregator decides which sources to read at all. Discovery mines the domains that brief-grade articles link to: a domain cited often by good outlets becomes a candidate, and a candidate gets a one-week trial. Papers link the packages they ship. So pypi.org was “cited 7x by 2 outlets”, registered as a candidate, and put on trial one morning at 05:49.
Its feed is every upload to the index, a hundred per fetch. The 08:11 run pulled a hundred package READMEs, the value triage scored six of them brief-grade, and a source on trial ranks at full weight from its first fetch. By 08:11 the brief had six package version bumps in its top fifty, two in the top five, and the three-item summary my glasses show carried one at number two. Two hours and twenty-two minutes from trial to front page.
Left alone, it would have passed the trial. A firehose produces brief-grade rows every fetch; that is what a firehose is. The denylist named artifact hosts but no package registry; it bound only at intake, so listing the registry would have left the running trial where it was; and the brief had no roster gate, so a shed source’s rows kept ranking until they aged out of the window. Three fixes, and the blast radius against the live roster was the package rows alone.
The same word twice
Both ends of the pipeline had read a link as an endorsement. The merit engine saw a paper point at a repository and paid it for shipping code. The discovery engine saw papers point at a package index and took that for a recommendation. A citation is neither. It says “this exists and I used it”, which is a different claim from “I made this” and a different claim from “read this next”.
The mistake survived because it flattered in both directions. A paper with a long bibliography looked like a paper showing its work. A domain every paper links to looked like a domain every paper vouches for. The scores went up, the ranking stayed plausible, and nothing was wrong except the meaning of the arrow.
The 470 papers in the window that actually shipped code still get paid for it. Four hundred and sixty-eight of them, which is what reading only the body costs.