OpenAI introduces BioBasicsBench to test AI agents in wet labs

A benchmark built around messy laboratory evidence

BioBasicsBench is intended to address a limitation in benchmarks built around charts and images selected for scientific papers. OpenAI says everyday wet-lab work often involves failed experiments, ambiguous images and conflicting observations, along with the troubleshooting needed to understand them.

The benchmark covers five areas: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Its tasks range from reading the molecular weight of a DNA sample in a gel to performing detailed assay analysis.

The inputs preserve the forms of evidence used by wet-lab scientists. They include gel and blot images, plate photographs, microscopy stacks, sequencing reads, plate-reader exports and videos of bench work. A task can combine several types of data, supported by reference materials such as ladder charts and plasmid maps.

Scientists reviewed every task

Scientists created the tasks using data from their own experiments and paired each question with expert-written grading criteria. Each task was reviewed by a Ligo scientist who had not written it, an external PhD-level scientist working in a wet lab, and a final internal reviewer.

Agents return a structured answer containing the requested conclusion, measurement or recommendation, plus supporting reasoning where required. The scoring system assigns credit to individual rubric criteria, allowing the benchmark to separate a correct conclusion from a partially correct analysis. Depending on the task, the criteria assess conclusions, numerical results, reasoning or recognition that the evidence cannot support a conclusion.

OpenAI evaluated seven models across 37 model and reasoning-effort settings. Each setting made five attempts on all 100 tasks, producing 18,500 results. At the xhigh reasoning setting, GPT-6 Astra achieved a mean reward of 66.7%, compared with 43.8% for GPT-5.6 Sol.

Results point to progress, with gaps remaining

At xhigh, GPT-6.1 Sol recorded a mean reward of 62.0%, while GPT-6 Luna scored 38.3%. GPT-5.5, GPT-5.4 and GPT-5.2 recorded 35.4%, 33.2% and 31.2%, respectively. OpenAI reports mean reward as the average task score, expressed as a percentage of available credit.

OpenAI says GPT-6 Astra improved on GPT-5.6 Sol even at low reasoning effort, and says the results suggest stronger native multimodal capabilities may be important for everyday wet-lab analysis. One example involved cells and two overlapping dye detection windows. OpenAI says Astra recognized that brightness alone could not establish whether cells were dead and called for better controls to separate the signals.

The company says the results show progress in routine lab analysis, but also room for improvement in interpreting images, checking experimental controls and applying experimental context. For researchers, that means BioBasicsBench is aimed at testing whether AI systems can handle the ambiguity and failure modes found in ordinary experiments, rather than only polished evidence prepared for publication.