Rebellions targets 4-bit reasoning errors with its ReQAT method
Rebellions and Hanyang University have presented ReQAT, a quantization-aware training framework designed to help reasoning models retain accuracy when weights, activations and the KV cache use 4-bit formats. The work is based on a paper presented at ICML 2026 as an oral presentation.
- ReQAT comes from Rebellions and Hanyang University.
- The framework targets 4-bit weights, activations and KV cache quantization.
- Rebellions identifies low-entropy tokens as especially vulnerable to FP4 errors.
- Trace-aligned training raised R1-Qwen-14B AIME accuracy to 67.29% in one reported setup.
- The code and training recipe are available on GitHub.
Why 4-bit quantization remains difficult
Quantization represents model weights and intermediate values with fewer bits, reducing memory use and bandwidth demand while potentially increasing compute throughput on supporting hardware. Weight-only quantization is already widely used, but extending 4-bit precision to activations and the KV cache creates a larger accuracy challenge.
Rebellions says this matters because low-precision hardware only lowers the cost per token when a model can still produce acceptable answers at that precision. The company builds AI inference infrastructure, and says the value customers receive depends on both the hardware and a model trained and quantized to run on it.
The challenge is especially pronounced for Large Reasoning Models, which generate long chains of thought. According to Rebellions, a single math problem can produce a trace of more than 16K tokens in models distilled from the DeepSeek-R1 family. Standard post-training quantization can substantially reduce math reasoning accuracy when such a model uses NVFP4 for weights, activations and the KV cache.
The source of the accuracy loss
Rebellions analyzed more than 1.5 million generated tokens by measuring the entropy of the next-token distribution. It found that roughly 80% of a reasoning trace consists of low-entropy tokens, including digits, variable names and operators. Higher-entropy tokens include discourse markers such as Then, Let and Hmm, where the reasoning path can branch.
The company reports that FP4 increases probability mass outside the top prediction at low-entropy positions. Under greedy decoding, that change has limited effect, but temperature sampling can increase the chance of an incorrect token across thousands of positions. Rebellions says one incorrect digit or operator typically invalidates the rest of a derivation.
In controlled experiments, routing only low-entropy predictions to BF16 recovered a large share of lost accuracy, while routing only high-entropy predictions to BF16 recovered little. Adding Gaussian noise to logits produced a similar result. The pattern held across R1-Llama-8B, R1-Qwen-7B, R1-Qwen-14B, Qwen3-4B and Qwen3-8B.
ReQAT focuses training on fragile predictions
ReQAT, or Reasoning-centric QAT, combines BF16 fine-tuning with quantization-aware training aligned to reasoning traces. Its Trace-Aligned QAT procedure first fine-tunes a model on a reasoning dataset, then applies QAT with quantization noise to a subset of the same traces.
Rebellions says this alignment helps training updates reach low-entropy predictions that have become fragile under quantization. On R1-Qwen-14B under MXFP4 W4A16, using a 350M-token budget, two-stage training without trace alignment raised AIME accuracy from 62.29% to 62.60%. Trace alignment raised it further to 67.29%.
The paper also introduces Selective Entropy Minimization, which adds an entropy-minimization term to the supervised fine-tuning loss. The supplied description ends while explaining this component, so it does not provide the method's full specification or a final W4A4KV4 result.
The code and training recipe are available at github.com/aiha-lab/ReQAT. The work is intended to address the model-side limitation that can prevent low-precision inference hardware from delivering its potential efficiency on reasoning workloads.
