benchmarks
AI21 says post-training pushed Nemotron 3.5 Super past frontier quality
AI21 says it has post-trained NVIDIA Nemotron 3.5 Super VL to outperform Claude Opus 5.5 on Deel’s global payroll benchmark, while costing 50x less per 1M tokens. The…
Ai2's StateBench Shows Frontier Models Struggle to Track State Over Long Inputs
Ai2 has introduced StateBench, a benchmark that measures how well language models keep track of entities as they change across long documents. According to Ai2…
Google tests cheap classifier to route Gemini tasks and cut agent costs
Google's developer blog argues that cost per successful task, not price per million tokens, now decides the economics of coding agents. In a 50-question benchmark, a…
Hugging Face publishes CWE-bench v2 with a public leaderboard
Hugging Face has published CWE-bench v2, a new version of its CWE-bench benchmark, under the title "CWE-bench v2: More coverage, quality, and transparency." A…
Krea 3 image editing model targets frontier quality at 8 seconds per edit
Krea has published the technical report for Krea 3, a foundation image-to-image model that the company says combines frontier-level quality with fast generation and…