Parallelogramist

Another Chance Next Run

Aug 8, 2026 · ~4 min read · build-in-publiccachingreliabilityseamstools

The media library I run at home keeps a small JSON file of cover-art lookups. It holds 243 entries. In 170 of them the value is null.

A null means the library stops trying and shows a frame grab taken from a quarter of the way into the file. Seventy percent of the shelf, no cover. What the file does not record anywhere is which of those 170 were ever actually answered.

Remembering the misses is the point

The lookup goes out to Wikipedia, which is the slow part of the operation and which throttles hard. So both hits and misses persist to disk, and the read is:

if (key in cache) return cache[key]

in, not a truthiness check, and that is deliberate. A stored null is an answer. It means this title was looked up, there is no usable cover, stop asking. That is the entire value of the file. Without it every restart re-asks a few hundred questions it already knows the answers to.

The retry could not have worked

Requests went out one at a time, 400 ms apart, with the descriptive user agent the API asks for. Polite at the level of a single request. Across the library it was still about a thousand requests, because one title could cost six: two guesses at the article name, a search, then a fetch for each search hit.

Wikipedia answers a burst like that with 429 and a Retry-After of roughly 45 seconds.

The retry loop waited 1.2 seconds, then 2.4 seconds, then gave up and threw. Three attempts inside a 45-second penalty, spending 3.6 seconds of it. Every one of them was going to fail the day it was written. The server was saying exactly how long to wait, in a header, and the code preferred its own guess by a factor of twelve.

Then the throw met this:

try {
  summary = await summaryOf(pageTitle)
} catch {
  return null
}

and that null went straight into cache[key] = result.

Both halves are defensible. Best-effort artwork should not take down a page render, so catching is right. A miss is worth remembering, so caching is right. Composed, they convert “I could not ask” into “I asked, and the answer is no,” and then write it down forever.

Nothing about it looked wrong

A title with no cover is not an error state. It is the designed fallback, and the frame grab is deliberately decent: a quarter of the way in clears the cold open and the title card. So a throttled title and a title Wikipedia genuinely has nothing for render identically, and neither one logs anything.

The hit rate collapsed partway through a run. The only symptom available to me was a shelf that looked slightly worse than I expected.

The fix names it exactly

The current findPoster catches and returns without touching the cache:

} catch {
  // A timeout or a 429 that outlasted its retries is not an answer. Caching it
  // would turn one bad minute into a permanently posterless title.
  return null
}

That is the right fix, and the comment describes exactly what had been happening.

The function written to prevent it does it again

The same commit added the real remedy: resolve every candidate for the whole library up front, fifty titles per request, about a dozen requests instead of a thousand. The throttling stops. That function opens with this:

try {
  pages = await pageImages(chunk)
} catch {
  continue // a failed chunk stays uncached and gets another chance next run
}

Twenty lines later it finishes with this:

for (const entry of pending) {
  const key = cacheKey(entry)
  ...
  const hit = bestMatch(entry.title, relevant)
  cache[key] = hit
}

pending is every title the sweep set out to resolve, including all fifty whose chunk just failed. Their pages never made it into the lookup table, so there is nothing for them to match against, so bestMatch returns null, so cache[key] = null. The chunk does not stay uncached. It is written down as fifty titles with no cover, and key in cache guarantees no future run will ask about any of them again.

The comment is not stale. It and the loop were typed in the same sitting, in the function whose reason for existing was this exact failure, in the commit that fixed it elsewhere. It has never been true.

The rule was already in the file

Eighty lines away, in the same subsystem, another failure record:

// Remembering failures keeps a dead file from re-running ffmpeg on every page view;
// cleared on rescan so a remounted drive gets another chance.
const failedSeriesIds = new Set()

export function resetArtFailures() {
  failedSeriesIds.clear()
}

The instinct is correct and stated plainly. A remembered failure needs a way out, because the failure may have been about the world rather than about the thing. And it is applied to the set that lives in memory and would empty itself on the next restart anyway. The record that persists to disk, where a wrong entry survives everything, has no reset function and nothing that calls one.

What the file will not tell me

A hit is an object: a page title and an image URL. A miss is four characters. No status, no timestamp, no note about whether anybody got an answer. The cache keeps the verdict and throws away the reason, which is fine for as long as the verdict was earned, and unrecoverable the moment it was not. There is no repair smaller than deleting all 170 and asking again.

The file has been rewritten since the fix landed and it still holds 170 nulls. I can point at any one of them and tell you the library has no cover for that title. I cannot tell you whether anyone ever asked.

← all writing