Never Collected, Always Green
Written later and filed to this date, as part of filling in every day of the year.
The test runner in my ops repository collects suites by walking packages: wherever it finds a package file, it runs the tests beside it. The fleet supervisor has no package file. Neither do the zero-dependency automations, the medic, the health checks, the shared library they import. Neither, until an afternoon in October, did a family of tool suites written as plain modules. A green run of the runner had been reporting on none of them. The comment above the step that finally ran the fleet’s suites says the other half: agents are forbidden from editing the fleet directory, which is exactly why the gate for it has to live somewhere they can reach.
Somewhere is scripts/verify.sh, and the rule that put it there arrived at the end of August, out of a review of what sessions did instead: long ad-hoc chains of commands, a test here, a type check there, a service probe if anyone remembered, each session composing its own and no two alike. The rule is one script per repository, the repository’s whole verification in a single invocation: the suite, the type check in the repository’s own toolchain, the lint, the service health. Every step prints PASS or FAIL with its name and its seconds. A failing step prints its last forty lines. The script exits non-zero on any FAIL, so a dispatch or a hook can sit behind it. The nine repositories the fleet was staffing that day each got one, four of them the same afternoon, and a new repository gets one at scaffold time.
The step list is the interesting part, because it is where the runner’s blind spots became visible. The ops repository’s script has seven steps. The first is the runner. The second runs the tool-module suites the runner did not collect. The third is the registry coherence check. The fourth runs the fleet’s suites by naming their files directly, and the fifth does the same for the automations. Each of those steps exists because a green run had once meant less than it said, and the fix was not to teach the runner but to add a line that could not be missed to a list that is read top to bottom.
The last two steps are classed WARN rather than FAIL. The document-size budgets and the service-reload coverage both report in the same format, and neither gates the exit, because an over-cap document is a standing choice of the operator and a service with no reload row is a registration to make, not a regression this run caused. The summary says so in one line: result PASS, seven steps, with the warnings listed above it. A script that hid them would be lying; a script that failed on them would teach people to skip it.
verify.sh: one script per repository that runs every check once, prints PASS or FAIL per named step, exits non-zero on any FAIL and never on a WARN, so that a green result means every suite in the tree, including the ones no runner collects.