Home Movies

live · design note

This page is one idea from this project, explored in depth. Source status: private system: only a high-level description is safe to publish. Withheld: It catalogs personal family media, so the code, the library, and where it runs stay private. The design is described here at the level of mechanisms, with a made-up evening as the worked example.

A self-hosted media server for the family's home movies: auto-organizes the library, transcodes on the fly, and streams to any browser. Kept off the public internet on purpose.

A home-movie library, served the way a streaming app would. A scanner walks the drive, groups files into a series and episode structure, and separates the formats a browser plays natively from the ones that need work. The second group is remuxed to HLS on demand. Reachable on the house network, or over Tailscale from anywhere else, and deliberately nowhere past that: this one is for the house, not for strangers.

This page is a design note. The code and the library are private, and nothing below links to either. What follows is the shape of the system, one trade-off, and the operational lesson it taught, written so that someone building their own box can take the mechanisms without the media.

The shape of it

drive  ->  scan  ->  library cache  ->  browser
                                          |
                        native format?  --yes-->  byte-range stream from disk
                                          |
                                         no
                                          |
                        one hardware encoder  -->  HLS segments  -->  player
                        (with a queue in front of it)

• Scan once, serve from the cache. The scan walks the media roots and writes one cache file, and the server boots from that file, so the drives can be asleep and the library still opens. A rescan that finds nothing never replaces a library that had something, because a sleeping drive looks exactly like an empty one.
• Play the original when the browser can. Files in a container the browser already understands stream straight from disk with byte-range requests. Nothing is transcoded that does not have to be.
• Everything else goes through one encoder. The box has a single hardware video encoder. Two concurrent encodes starved each other, so there is one job at a time and a queue in front of it. Output is fragmented-MP4 HLS at the original quality by default (a stream copy when the codec is already playable, a re-encode at source resolution otherwise), with a low rung produced in the same pass so a weak link degrades instead of stalling.

The trade-off: one encoder is a scheduling problem

With one encoder, every policy question is really “who gets it next”. The rules that shipped:

1. A real play request wins over everything. A speculative job (the next episode, prefetched during playback, or the top of Continue Watching warmed up when the library opens) runs only while the encoder is idle and is preempted the moment someone presses play.
2. Speculative work that nobody claims is discarded after a few minutes, so a guess never holds the encoder or the disk.
3. Finished segments live in a size-capped cache that evicts the least recently used, so the second viewing of anything starts instantly and the drive never fills.

The alternative, transcoding the whole library ahead of time, would trade the queue for a library-sized encode of things nobody may ever watch. On a box with one encoder and a large drive, on-demand plus a warm cache was the better bet, and the warm-up rule recovers most of the instant start that a pre-transcoded library would have had.

The operational lesson: a job needs to know who is watching

The first version stopped a transcode when a viewer left. The page sent a departure beacon; the server stopped the encoder and deleted the segments. Two failures came out of that one rule, both told in the essay Every Retry Made It Slower: a reload counted as leaving, so the segments vanished out from under the same viewer, and a cancel stopped the encoder it could see while abandoned subtitle passes kept reading the same file, so each retry was slower than the last.

Both had the same missing fact: the server did not know how many viewers a job had. The fix is a lease. Every status poll from a tab renews that tab’s lease on the job. Leaving releases one lease and never touches the job. A job ends only when its last lease has been gone for a grace period (long enough for a reload to come back and reclaim its own segments), or immediately when a real play request needs the encoder, or on an explicit cancel, which still spares a job someone else is streaming.

A made-up evening shows the rules working together. One tab starts an episode; the encoder begins and segments appear. A second tab in another room opens the same episode and joins the job already running for it instead of starting a second encode. The first tab reloads; its lease lapses and returns inside the grace window, and the segments are still there. Meanwhile the next episode is being prefetched, and when a third person presses play on something else, that prefetch is preempted and the real request gets the encoder.

What this is not

It is one box with one encoder: two people playing different files that both need encoding will queue, by design. Nothing here is exposed to the public internet, and nothing here is packaged for anyone else to run; the write-up is the artifact. Titles, artwork, and metadata come from public sources, but the library they describe is a family’s own video and stays where it is.