Goodfire researchers identified a single "addition module" in layer 18 of Llama 3.1 8B that handles both arithmetic and cyclic reasoning about months and days of the week. The findings, published in a technical paper earlier this week, suggest the model reuses one piece of internal machinery across surprisingly different tasks.
Numbers, but as circles
The intuitive guess is that a language model would store numbers along an internal number line, or maybe in some binary encoding. According to the Goodfire post, neither is true. Llama represents each number using a stack of circles in activation space, where each circle encodes the number modulo a small base.
Take 17. On the mod-2 circle, it sits at position 1. The mod-5 circle puts it at 2. Mod-10, position 7. The mod-100 circle reads 17 directly. Several circles, one number, all at once.
This is a Fourier decomposition, which Goodfire compares to a residue number system. Prior work by Kantamneni and Tegmark already showed that GPT-J does something similar, so the circular encoding itself isn't new. What's new is what Goodfire claims the model does with it.
Parallel modular arithmetic
When Llama needs to add 6 and 8, the addition module solves the problem on each circle separately. Mod 2: 0 + 0 = 0. Mod 5: 1 + 3 = 4. Mod 10: 6 + 8 = 14, which becomes 4 mod 10. Stitch those modular results back together, and the model reads off 14.
Each circle owns a different subproblem, run in parallel inside a single forward pass. Individual neurons in the module specialise too: some only read from and write to the mod-2 circle, others to mod-5, and so on.
Reuse
Llama uses the same module to answer "what is two days after Friday?" and "what month is sixteen months after August?". The model converts August to 8, runs 8 + 16, takes the result mod 12, and produces February. One module, several jobs.
This is the kind of finding interpretability researchers actually care about. Plenty of mechanistic interpretability work has identified circuits that do one thing in one context. A reused, general-purpose arithmetic module is closer to the kind of "function" you'd expect in software, which is the long-running pitch for interpretability as a practical engineering discipline.
How much of this is real?
The team also tried causal interventions: nudging the circles in activation space and checking whether the model's answer changes accordingly. It does. But the paper notes the steering experiment required pushing circle magnitudes "many times the naturally occurring value" to produce clean shifts in the output. The authors describe their own story as "compelling, but incomplete."
That's a useful caveat. The correlational evidence for the addition module is strong: neuron firing patterns line up cleanly with modular arithmetic across tasks. The causal evidence works, but only after amplifying the signal. Whether the module dominates Llama's actual behaviour at normal activation levels, or just contributes to it, is the open question.
What's next
Goodfire's broader Neural Geometry Series, launched in early May, frames this addition module as one example of a larger pattern: model representations sitting on curved manifolds rather than flat linear features, and computations performed over those manifolds. The lab says it's working on methods to discover similar mechanisms at scale, with more posts in the series listed as upcoming on the research index.
