Ask Again in 229 Sessions

The first thing my paper portfolio said, the night it learned to compare itself to the index, was that it was losing. Four hours later I wired that number to post on every sync to the feed my dashboard and glasses read. Six hours after that, the same tool learned to say the number meant nothing, and would go on meaning nothing for about 229 more trading days.

The meter

The portfolio is an experiment. A research tool of mine studies factor strategies and issues a verdict, and since 19 August a paper account has held whatever the verdict recommends. No real money moves. The point is to find out whether following the tool beats doing nothing clever.

“Doing nothing clever” needed a number, so the change replays every fill the account ever made into VTI, a fund that holds the whole US market: the same dollars, on the same days. First reading: 1.63% behind over seven weeks.

The status event I built for it reports on the plumbing, never the result: error if the broker could not be reached, ok if it could. A losing book is an ok sync. Nothing would turn red. The number would just sit one glance away, and a number one glance away gets glanced at.

The line under it

The next morning the meter grew a second line. Annualized, the gap is -10.3% a year over 35 sessions. The 95% range around it runs from -43.5% to +23.0%. Beside it: luck cannot be ruled out. Then the part I keep rereading: a gap this size, at this spread, becomes distinguishable from luck at about 264 sessions. Thirty-five down, 229 to go, roughly 0.9 years.

My first reaction was that the range was too wide. A textbook 95% interval from the same 35 sessions runs from about -28% to +8%, still straddling zero, and it would call a gap this size real after about 110 sessions instead of 264. The commit message explained, in one sentence, why the tool refuses the shortcut: the number is re-read on every sync, and “a fixed-sample interval checked daily would eventually exclude zero by chance alone.”

Looking is a trial

I wanted to know how soon “eventually” is, so I simulated it: 20,000 books with no edge at all, the same day-to-day spread as mine, checked after every session from the twentieth on.

With the textbook interval, 15% of those edgeless books had been declared ahead or behind at some point by session 35, which is where mine is now. Within one trading year it was 34%. Within three, 43%. Each check is a fair 95% test. Run one every morning and they compound toward a coin flip that a strategy with no edge will, on some ordinary Tuesday, look like it has one, or look like it is failing for reasons you can then go and find.

The tool uses a confidence sequence instead, an interval built to hold its 95% across every session it will ever be checked at, not only the one you happen to stop at. In the same simulation it raised a false alarm on 2.2% of books within a year and 3.2% within three. (The simulation assumes tidy daily returns, which real ones are not; that is also why the tool shows no interval at all before 20 sessions.) Its price is the 150-odd extra sessions it makes me wait.

I would never have chosen that trade on purpose, because I never thought of glancing as a choice. The textbook interval answers a question asked once, at a sample size fixed in advance. Nobody asks a dashboard once. Wiring the number to my glasses was a promise to run the test every morning, and the honest width of the answer grew to match.

The timer that posts it is still an older installed copy, so the glasses have not shown the number yet. When they do, it will arrive with its second line already attached, saying what it will say every morning until about this time next year.


← all writing