Four Minutes Early
The timer fires at 03:26. It is scheduled for 03:30.
Not once. Every night, for as long as the journal goes back. I noticed by accident: I was grepping the logs for an unrelated build failure, and the timestamps kept opening with 03:26, and some part of my brain that files things without asking finally flagged it.
My first thought was jitter. The unit has a randomized delay on it, like most of my nightly jobs, so the firing time is supposed to wander. But randomized delay only adds time: it schedules a window after the calendar spec, never before. A timer can be late. A timer is not supposed to be early. Four minutes early, to the second, every night, is not what wandering looks like.
Second thought: clock drift. The box syncs time like everything else, but drift happens, and drift was a comfortable explanation. I checked the sync status. Offset measured in milliseconds. The clock is fine. I want to be clear that I checked this twice, because the clock being fine is the part I keep coming back to.
Third thought: I wrote the unit wrong. The calendar spec says 03:30. I read it, then rewrote it anyway, the way you re-save a file you haven’t changed, and re-enabled it. That night it fired at 03:26.
Fourth thought, the deflating one: I was misreading the journal. Maybe those lines were the service starting for some other reason and the timer was innocent. So I asked systemd directly. Its own bookkeeping says the timer elapsed at 03:26. The same bookkeeping that holds the 03:30 instruction. It agrees with itself and disagrees with its instructions, and it does this once a night with perfect manners.
There is a boring explanation. There always is. Something in the dependency graph pulling the unit in early, a stale drop-in, an override I made in March and forgot. I’ll find it on a weekend at some point and feel stupid, and that will be the end of it.
What I keep snagging on is the consistency. Drift wanders. Bugs are sloppy. This is exactly four minutes, every night, and nothing downstream of it is ever out of place.
I’ve filed it as drift. It fired at 03:26 this morning. I checked before writing this.