Note4 November 2026
Why I post findings before peer review
Interpretability moves fast and much of it isn’t settled. Posting checked findings early, clearly labelled, seems more useful than holding them back for a year.
Interpretability moves fast. A lot of what we think we know about how models work isn’t established yet, and a result can be overtaken by the time it clears review. Holding findings back until then helps nobody, so I post them here once they’re written up and checked.
What “checked” means
Before an entry goes up:
- the study’s question, predictions and analysis plan were frozen before the results came in;
- every number in the paper was generated from the study’s saved outputs;
- the code, the per-run outputs and the frozen protocols are public, so anyone can rerun the checks.
What it doesn’t mean
It doesn’t mean anyone else has reviewed it. Every entry says where it stands, in red ink, at the top: working paper, preprint, under review or published. When a paper gets through review, its label changes and the entry links to the published version. Until then, read the conclusions as provisional.
Why bother with the process notes
Each entry ends with a short record of how the study went: when the protocol was frozen, what was run, what failed, and what changed between versions. In a young field the route to a result says as much as the result: which checks it passed, which it didn’t, and what would have to be true for it to be wrong.
If you find a problem, open an issue on the paper’s code repository. I’d rather know.