OpenAI tested AI agents on real wet-lab data

OpenAI announced a new test set called BioBasicsBench on November 6, 2026. The 100-task set was prepared together with enzyme design company Ligo. The goal is to answer a simple question: can AI agents really read the raw data that comes out of routine lab experiments?

OpenAI reports the results using average reward, which shows what percentage of the total possible score for a task was earned. At the company’s highest reasoning setting, GPT-6 Astra reached 66.7% (95% confidence interval 60.3–73.1). GPT-5.6 Sol, meanwhile, stayed at 43.8% (37.6–50.0). While Astra produced an average of 4,704 output tokens per task, Sol spent 9,099 tokens.

The tasks come from real experiments

OpenAI’s criticism is that current benchmarks are generally built around charts and visuals selected from scientific papers. But day-to-day lab work also includes failed experiments, ambiguous images, conflicting observations, and rounds of troubleshooting.

The tasks in BioBasicsBench ask agents to interpret experimental data, catch errors, and recognize when the evidence is not enough to support a conclusion. The scope spans protein methods, imaging and cytometry, measurements and assays, nucleic acid methods, and molecular cloning. Inputs include gel and blot images, colony plates, microscope stacks, sequencing reads, plate reader outputs, bench photos, and protocol videos. Some tasks combine multiple types of evidence and reference pages such as ladder plots or plasmid maps.

Scientists derived the tasks from their own experiments, and they also wrote the scoring rubrics. Each task went through a three-stage review: a Ligo scientist who did not write the task, an external PhD-level lab expert, and a final internal check covering the prompt, evidence, rubric, and a sample model answer. The evaluation was run across 37 models and reasoning settings, with five attempts per task, producing a total of 18,500 results.

Where the gap between Astra and Sol shows up

The examples shared by OpenAI suggest that the difference lies less in raw observation and more in reading the experimental context. In one cell imaging task, Sol classified the cells as dead based on the dye signal. Astra, however, noticed that overlapping detection windows could cause interference from a second dye and asked for control groups that would separate the signals.

In one DNA cloning task, Sol treated the result as reliable because it saw no colonies on the negative control plate. Astra noticed that the plate was labeled for kanamycin rather than the ampicillin used in the experiment, and rejected it as a control. In a cell image with shifting focus, Sol looked for a biological explanation, while Astra identified the image as a z-stack taken at different depths.

A high score does not mean reliable judgment

OpenAI explicitly says that a higher score still does not mean reliable lab judgment. In a colony PCR task, Astra selected wells 4, 6, 7, and 8, assuming the larger PCR product indicated successful cloning; the reference interpretation pointed to 5, 9, and 10. That mistake would have led to purification of the wrong plasmid.

In other examples, correct observations do not carry through to the final answer. Astra generalized that vacuoles are usually darker than the surrounding tissue, even though lighter-colored vacuoles were present in the same image. In a colony counting task, it failed to separate close neighbors, missed visible colonies, and marked empty agar; compared with the reference, it overcounted by about 12%. Because plate volume and dilution were fixed, that deviation also inflated the calculated DNA transfer efficiency by the same amount.

OpenAI says tracking the development of these capabilities will also help inform its assessment of biosecurity and biosafety implications. The bar set by the benchmark is higher than the total score: the agent needs to carry correct observations through to the conclusion, question controls, and avoid making judgments where the evidence does not support them.

Source: Unite.AI