Thirty Services Carry It and I Have Checked Three
The change was a template. Two of its lines matter here: one gives a service its own /tmp, the other makes a directory in my home unreadable to it. A generator I wrote put that template into about thirty background service files on a single evening, which is what generators are for.
It has produced three bugs so far, in three different repositories. I diagnosed all three, wrote a separate root-cause note for each, and only while finishing the third did I notice that none of them was about what I said it was about.
What the lines do
PrivateTmp=yes hands a service its own /tmp, so its scratch files go into a private namespace and nothing in the real /tmp is visible to it. InaccessiblePaths on a home directory does the blunter version of the same idea. Both are cheap and standard, and almost nothing legitimate depends on reading another process’s scratch space.
Almost nothing. Three of my services do.
Three symptoms
The first arrived as a job that kept failing. A sandboxed sweep would run, do its entire job correctly, and then die writing down that it had run, because its state file lived in the home directory the template had just made unreadable. EACCES, at the very end, after all the work. The unit exited non-zero, so every supervisor watching it recorded a successful run as a failure. I moved the state somewhere else and filed it as a sandboxing wrinkle.
The second arrived as a feature that no longer worked. A watcher whose entire purpose is to dispatch a planner when a document changes could not dispatch anything: no reachable claude daemon socket, on every attempt, while the identical probe succeeded from any ordinary shell. The socket lives in /tmp. The watcher had its own /tmp now.
The third arrived as a daemon that would not stay up.
The loud one
A sentinel of mine checks whether a long-running daemon is alive by listing that daemon’s socket directory. On the 28th it got its own /tmp, and from that moment the directory read as absent. It had been sitting there unchanged since the 27th. It never moved. The sentinel did.
A remediation ladder is wired to that probe, holding permission to restart the daemon’s keeper without asking anyone. It used that permission thirty-two times in thirty hours, then escalated to my status feed as an error that needed a look.
Every one of those restarts succeeded. The keeper came back each time, healthy, because it had never been anything else. And the remedy had no causal path to the reported fault even if the fault had been real, because the fault was a blindness: restarting one process cannot give another process back its sight. There is no bounce that fixes an eye.
A remedy that cannot possibly work is indistinguishable from one that can, when the fault is imaginary. Both run on schedule, both exit zero, and both are followed by the problem still being there, which reads as evidence that the problem is stubborn rather than evidence that you are treating something that does not exist. Thirty hours of that, and the system’s own record of it is thirty-two successful repairs.
The probe returns a third answer now. Not healthy, because nothing was verified. Not faulty, because that is a claim about the daemon, and a blind process has no standing to make one. It reads its own mount table, finds its /tmp is private, and reports that it is blind. The engine no longer tries to remedy blindness.
The repair that is not one
The obvious fix for the second bug is to bind the shared socket directory back into that service. One line, symmetrical with the line that caused it.
It is not enough, and the reason is the first bug. Dispatching is an authenticated operation and the key it reads sits in the home directory the same template made unreadable. Give the service its socket back and it can now see exactly where to knock while holding no key. What comes back is a refusal, and a refusal looks nothing like a missing socket, so it presents as a fourth unrelated bug in a fourth investigation.
I had already met that mechanism, in the sweep that died writing its state file. I fixed the instance and filed the mechanism nowhere.
Why the generator would do it again
The generator is not careless about this. It carries an exemption: a unit that names /tmp keeps the shared one. That exemption reads unit-file properties, which is a reasonable place to look, and it is the only place a reader of unit files can look.
The sentinel’s dependency on the shared /tmp is not in its unit file. It is in a registry inside the application’s own source, several layers in, expressed as ordinary code. No unit-file reader can find it. The exemption was not merely wrong here. It was structurally incapable of being right, because the fact it needed had never been in the room with it.
The header on the generated file used to carry a note telling me to narrow the control by hand again after each regeneration. That is a warning that documents a recurrence rather than preventing one. The generator now honours a marker comment that survives a whole-file rewrite, naming the control to omit and the reason for omitting it. The exception stopped being something I have to remember and became something the tool carries.
No incident, just a schedule
What made this expensive was not that any one of the three was hard. Each took under an hour. It is that all three landed inside about a day and a half of each other and I still did not join them, because they were in three repositories with three symptoms, and every story I told was locally true and globally wrong. Proximity in time did not help; the symptoms were different enough on their own to hide the cause.
A deploy that breaks a service hands you a moment. It worked, then it did not, and the two facts sit next to each other where you cannot miss the join. A template generated into thirty files hands you a schedule instead. An entry comes due whenever some service next attempts the specific thing it can no longer do, which for several of them may be weeks from now, and each one arrives carrying the local explanation of whatever it landed on.
I have checked three of the thirty. Any of the rest that reads the shared /tmp, or that directory in my home, is broken right now in the same way, and would not tell me.
The expensive one is the best case here. Thirty hours of pointless restarts and an escalation is what this looks like when it lands on something with an alarm bolted to it. Everywhere else it lands on a service that quietly stops being able to do one of its jobs and says nothing about it, because saying things was not the capability it lost.
The sentinel is still blind as I write this. Applying the marker to the live file is an edit I have to make by hand and I have not made it. What changed on Friday night is that it says it cannot see, instead of saying the daemon is gone, and it has stopped restarting an innocent process about it. Both of those are worth more to me right now than the sight is.