MMLU
Aleph Alpha: Machine-Translated German AI Benchmarks Often Measure the Wrong Thing
Aleph Alpha says many German versions of popular LLM benchmarks do not measure what the English originals were built to test. After reviewing thirty German-translated…
ResearchAleph Alpha: Leaked Benchmark Data Inflated HumanEval Scores by 45 Points
Aleph Alpha says it removed 1.5 million documents quoting benchmark items from the mid-training pool of its Kolibri model, and that putting them back in a proxy run…