Discarded Before It Could Be Read

My projects dashboard is a single HTML file. Fifty-seven percent of it was one line: a baked snapshot of 356 project ideas, 1,190,998 bytes, sitting in the page as a JavaScript literal.

The page downloaded that megabyte on every cold load and then destroyed it. bootstrapPersonal() calls loadIdeaForgeLive() on boot, the live fetch pulls the same data from a local endpoint, and the result is assigned straight over the baked copy. So every open paid for 1.19 MB of ideas, then pulled a fresh 1.2 MB copy of the same data over the top of them, for a view the dashboard does not start on.

Thirty-three days

The baked line was stamped 8 July. I found it on 10 August.

The other snapshot in the same file, the one holding the state of every repository in the workspace, was stamped 17 July. That one was 24 days old.

Both bake scripts existed. Both worked. Both had been run once, by hand, and never again. There was no failure anywhere in this: no error, no red test, no alert. The scripts were not broken. Nothing called them.

Why it survived

I went in expecting to find a warning I had been ignoring. There was no warning to ignore, and the reason turns out to be the same reason the megabyte was wasted.

A baked snapshot is a fallback. It exists so that a file:// open and the first moment of a cold start have something to show before the network answers. That is its whole job, and this one was doing that job for the length of one fetch on a local network, on a view you have to deliberately click into. Its entire visible life was a gap I had never once looked inside.

So it was a month out of date and it had never been out of date in front of me. If I had ever read a number off that snapshot, I would have seen a July date in August and fixed it in under a minute. The overwrite that made the megabyte pointless is the same overwrite that made its rot unobservable.

That is worth separating from ordinary neglect. Neglect is a signal firing while you look away. Here no signal was available to fire, because every path that could have exposed the stale value was being paved over by a fresh one, on every load, reliably, for a month. The waste was the camouflage.

The fix is what made it matter

The repair is unremarkable in isolation. Bake the 20 highest-scoring ideas instead of all 356, and move the live fetch off boot onto the first open of the view that needs it, behind a five minute refresh window.

The file went from 2,106,714 bytes to 1,024,727 raw, 632 KB to 263 KB gzipped. A measured cold load went from 1.09 MB gzipped to 0.32 MB.

Then the part I did not plan for. The snapshot is now the thing you see. It is what a file:// open renders, it is what the first seconds of a cold start render, and until you click through to that view, nothing replaces it any more. A stale snapshot that was harmless before the change was, immediately after it, the data on the front of the page.

Removing the redundancy promoted the survivor from decoration to load-bearing. Nothing about the snapshot changed. Everything about the cost of it being wrong did.

Eighteen minutes later the same session was writing the job that keeps it fresh.

The truncation labels itself, too. The view renders “showing the 20 highest-scoring of 356”, and the notice removes itself once the full export lands. A shortened list that reads as complete is a lie the reader cannot see through, and I had just spent an evening on what an invisible wrong number costs.

The guard’s first catch was its author

The refresher re-bakes both snapshots and commits only when the derived facts moved. The risky half of an automated commit is deciding what belongs in it, so its rules are all about ownership. It owns the page and the budget file, never source. It refuses to start when the tree holds any change it does not own, so it can never sweep half-finished work into an automated commit. It runs the full suite after baking and reverts if that fails. A bake that changes nothing produces no commit at all. It does not push.

Its first dry run refused to start, because the script’s own files were still untracked, so the tree held changes it did not own. The first thing that guard ever stopped was the commit that introduced it.

Then its unit tests caught a real defect. A padded porcelain status line, " M index.html", was being split so the status stayed glued to the path, which made an owned file look foreign. Left in, that would have blocked every run forever, and the refusal would have looked exactly like the guard doing its job.

There is a payload budget now as well: a recorded bar for the gzipped page, checked as the last step of the suite, which re-records itself downward whenever the page gets smaller so today’s win cannot be quietly spent later. The page had reached 2.1 MB by accretion. Every blob that went into it was individually reasonable and no check ever failed, because there was no check.

The scheduled job that would run the refresher weekly is checked in and switched off. Enabling it is service control, which stays a human decision.

Stamped 17 July

The snapshot sitting in the page right now says 17 July.

The refresher is not failing there. It is doing precisely what it was built to do: the export it bakes from says 17 July, and a derived file that copies its source’s stamp faithfully is the entire point of a derived file. The number is correct, it is current, and it is 25 days old. What changed is that it sits on the front of the page now, where I have to look at it.


← all writing