SynAgent Framework for Autonomous Experimentation: SynAgent pairs large language model agents with automated laboratory hardware to conduct scientific campaigns. The system constructs data analysis procedures dynamically instead of using fixed pipelines, applying verify-falsify testing to multimodal inputs like microscopy images and diffraction patterns. Across an 18-experiment test run, the framework identified thermal boundaries and crystallization parameters for LiCoO2 thin films while recording the underlying physical process.
TSBench 3D Transition-State Benchmark: TSBench tests the ability of LLM agents to assemble three-dimensional transition-state structures using structural editing tools, validating outputs through an automated quantum-chemical pipeline. An evaluation of seven frontier models on 78 elementary reactions showed that diagnosis-driven iteration raised aggregate success rates from 50.4% to 66.8%. Agent accuracy fell as reaction complexity increased, with errors frequently stemming from saddle points that connected to unintended reaction endpoints.
Self-driving laboratories can explore synthesis conditions autonomously, but their decision-making layer is typically a black-box optimizer, and the output is a set of optimized samples, with the measurements reduced to predefined scalar objectives and the reasons behind success left unarticulated. Here we present SynAgent, a framework in which large language model agents operate an automated experimental system and maintain an explicit, revisable understanding of the synthesis process as the campaign's primary output. Starting with no predefined analysis pipeline, SynAgent adaptively…
Understanding a chemical reaction requires mapping a symbolic reactant-product description onto the three-dimensional pathway through which atoms rearrange, yet chemistry benchmarks for large language models (LLMs) largely probe factual knowledge and text-based reasoning. Here we introduce TSBench, a benchmark in which an LLM agent uses structure-editing tools to construct three-dimensional transition-state (TS) guesses verified by an automated quantum-chemical pipeline, yielding a physics-grounded pass/fail verdict. Across 546 evaluations of seven frontier LLMs on 78 elementary reactions,…
Full transcript
Directing laboratory hardware to synthesize materials is one thing; developing an operational understanding of the physics during the process is another. That is our focus today on Agents in Research, examining autonomous tools in scientific inquiry. Here is the report.
Standard autonomous laboratories frequently rely on black-box optimization algorithms to search through synthesis conditions.
The difficulty with that approach is how those platforms compress measurements into predefined scalar targets. A system might isolate an optimized sample, but the physical rationale behind the outcome remains unspecified.
That specific gap led to SynAgent. Instead of operating as a black box, it pairs large language models with automated synthesis hardware to construct an explicit, revisable model of the physical process.
What changes in practice is the data processing. Rather than routing results through preconfigured pipelines, it generates custom analytical routines dynamically as measurements arrive.
That includes multimodal inputs like diffraction patterns and microscopy images, which pass through a verify-falsify procedure to confirm or rule out mechanistic explanations.
In an eighteen-experiment evaluation on lithium cobalt oxide thin films, the framework mapped crystallization parameters and thermal phase boundaries, updating its structured process understanding after each physical run.
Moving from physical synthesis to computational chemistry, TSBench targets a similar gap in how language model agents handle physical structures.
Existing computational chemistry benchmarks focus mostly on text-based knowledge retrieval or symbolic representations. TSBench requires agents to assemble three-dimensional atomic candidate structures directly using molecular editing tools.
Each candidate geometry is then tested through an automated quantum-chemical verification pipeline to evaluate physical validity.
Across seven frontier models tested on seventy-eight elementary chemical reactions, diagnostic feedback and iterative revisions brought aggregate success rates from 50.4 percent up to 66.8 percent.
Even with those tools, accuracy dropped as the mechanistic complexity of the reactions grew.
The dominant failure mode was locating saddle points on potential energy surfaces that connected to incorrect reactant and product states instead of the intended reaction pathways.
We will track further developments in autonomous scientific systems next week. From Agents in Research, thanks for listening.
