Fifty-Four Files, None of Them the Package
Written later and filed to this date, as part of filling in every day of the year.
What. A factor-research engine, three days old, whose entire data layer was missing from git: the as-of module that keeps a backtest from seeing the future, the store, the universe, the integrity audits, and every source adapter. Thirteen files, close to two thousand lines. The remote held fifty-four files and none of them were that package. A fresh clone could not import the project at all.
When it was noticed. On the third day, by cloning. Everything short of a clone said fine: the package was on disk in the working checkout, so every local test passed, and git has nothing to say about files it has been told to ignore, so every commit was clean and every push succeeded.
Cause. The ignore file carried a line reading data/, meant for a data directory at the repository root, where large and licensed vendor extracts live and must never enter git. Unanchored, that pattern matches a directory called data at any depth, and the source package has one: its data layer lives in a subdirectory named data. From the first commit on, git had filed the repository’s crux under artifacts.
Fix. Anchor the artifact ignores to the root, /data/ and /runs/, and add the package. The unanchored patterns for parquet, zip and sqlite files stay as they were, so real data artifacts under the package’s data directory remain ignored, which they should. Thirteen files added in one commit, titled for what had happened: track the package the ignore pattern had swallowed.
What guards it. A comment above the anchored line saying why it is anchored, and the knowledge that nothing mechanical can. An ignored file is not an uncommitted file. Status does not list it, a clean-tree check does not count it, and the push that follows has nothing to carry. Every check in that chain asks git what changed, and git answers truthfully about the files it knows. The one test that catches this is the one that was run on the third day: clone it somewhere else and try to import it. It was run by hand, and it is the only reason the package is in the history at all.