Richard Jeevash

Logogram

Change one activation. Watch what changes. A local-first workbench for causal experiments inside language models.

Load a model, set up a clean and a corrupted prompt, patch every attention head, and see the evidence behind each number. No account, no cloud, no telemetry.

Logogram on GitHubOn PyPIGetting started

Every run writes its own logogram. This one is drawn from the strongest head effects in the IOI example.

What it asks

How it finds out

The simplest experiment, on the IOI example that comes with Logogram, copies one attention head’s output from a prompt where GPT-2 small answers Mary into one where it answers John, and measures how far the answer moves back. Logogram does it for every head in turn and paints each effect onto a map of the model.

CLEANWhen Mary and John went to the store, John gave a drink to → Mary

CORRUPTWhen Mary and John went to the store, Mary gave a drink to → John

PATCHEDThe corrupt run, with head 9.9’s output copied in from the clean run.

Head 9.9 brings the answer back by about a sixth of the gap, so its effect is 0.17. An effect of 1 would bring Mary all the way back; 0 would change nothing.

Three workspaces

  • Explore. Look inside the model: its map, attention patterns, layer predictions and an SAE’s features.
  • Experiment. Set up prompt pairs, check the baseline, and see every choice that can change a number.
  • Evidence. Read each run with its statistics and per-prompt evidence, then compare runs or sweep them.

Every number has a paper trail

  • Traceable. The intervention, baseline, metric, n and confidence interval are always shown.
  • Recheckable. Check robustness reruns a sweep another way and flags the components whose conclusion changed.
  • Plain files. Experiments are folders you can commit, share and rerun.
  • One spec. The app and logogram run execute the same spec and give identical results on the same machine.

Methods and models

Activation patching, ablation, attribution patching, path patching, direct logit attribution, steering, SAE features, circuits and the logit lens, for the decoder-only language models TransformerLens 4 loads: GPT-2, Pythia, Llama, Mistral, Qwen, Gemma, OLMo, Phi and more.

Try it

uv tool install logogram
logogram

It opens in your browser. Start with the IOI example: 32 prompts, every head patched, about a minute on a CPU. Logogram is open source under the MIT licence.