HumanEval
AI21 Labs: Public coding benchmarks miss how developers really use agents
AI21 Labs says public coding benchmarks such as SWE-bench, HumanEval and Spider give a skewed picture of how developers use AI agents. After classifying seven weeks of…
ResearchAleph Alpha: Leaked Benchmark Data Inflated HumanEval Scores by 45 Points
Aleph Alpha says it removed 1.5 million documents quoting benchmark items from the mid-training pool of its Kolibri model, and that putting them back in a proxy run…