One Space Shifts Answer Probabilities by Orders of Magnitude, Aleph Alpha Finds
Where a single space sits in a multiple-choice prompt can change the probability a model assigns to the correct answer by several orders of magnitude, according to new research from Aleph Alpha. Accuracy, however, mostly stays stable, and the researchers trace the effect to a mismatch between tokenization in training and in evaluation.
- Placing one space in the prompt instead of the answer option lowers the correct answer's probability by several orders of magnitude, according to Aleph Alpha.
- The cause is tokenization: the canonical form appears about 2,300 times per million tokens, versus 0.03 to 0.10 for the alternative.
- In six of nine model-benchmark pairs accuracy changes by under half a point, but Qwen3.5 loses almost ten points on PIQA.
- Models also show label bias, such as B for Qwen3.5 and D for Gemma 4.
- Aleph Alpha backs Space-in-Answer as the default.
A tiny formatting choice with a large effect
A common convention in multiple-choice evaluations is to end the prompt with Answer: and read off the model's next-token probabilities for the option labels A, B, C and D. In ordinary text a space follows the colon, so the string reads Answer: B. That space has to belong either to the prompt (Space-in-Prompt) or to the answer option (Space-in-Answer).
A 2025 paper by Sanz-Guerrero, Bui and von der Wense, titled Mind the Gap, found empirically that Space-in-Answer scores consistently higher across a range of models and recommended it as the default. Aleph Alpha's authors, Prabhu Sivaprasad, Frank Schneider, Martin Reinhardt, Fabien Benureau and Tom Burns, now explain why.
The tokenizer is the cause
During pre-training, a model sees text such as Answer: B and learns to predict the next token. Aleph Alpha calls the resulting split the canonical tokenization. For the six modern tokenizers the team examined, the text splits into Answer, : and ␣B, so the model learns to predict the label together with its leading space.
On a one-million-document sample of DCLM-Baseline, the canonical form appears roughly 2,300 times per million tokens. The alternative, a standalone space followed by a bare B, appears about 0.03 to 0.10 times per million, a gap of four to five orders of magnitude.
Space-in-Answer reproduces the canonical token sequence exactly for every tokenizer tested. Space-in-Prompt does not: the prompt ends in a standalone space and the model must predict a bare B, a sequence it has practically never seen in pre-training. The authors call the underlying condition tokenizer composability, meaning that tokenizing prompt and answer separately and joining them should equal tokenizing the combined string. All tokenizers examined break it at the prompt-answer boundary.
Probabilities collapse, accuracy mostly does not
The team computed log-probabilities of the correct answer under both splits on MMLU, ARC and PIQA, using the base models Olmo-3-7B, Qwen3.5-35B-A3B and Gemma-4-26B-A4B. With Space-in-Answer, the values concentrate near zero, meaning the model puts most probability on the correct label. With Space-in-Prompt they are lower and spread wider, consistently across all models and benchmarks.
Accuracy is affected much less, because the drop hits all candidates similarly. In six of nine model-benchmark pairs, accuracy changes by less than half a point. Qwen3.5 is an exception, losing almost ten points on PIQA.
The researchers also note a label bias when options are scored as bare letters: models favor some labels, such as B for Qwen3.5 and D for Gemma 4. This affects a few benchmarks, mainly when the model is less confident.
What it means for evaluation
The authors say any quantity built directly on these likelihoods, for example the bits-per-byte of the correct answer, is also influenced by the choice of split. They conclude that Space-in-Answer remains the right default, since it matches what the model saw during training. For anyone comparing base models with likelihood-based multiple-choice tests, the finding is that a seemingly irrelevant formatting detail can change what the benchmark actually measures.
