홈 › Papers

COMPOSITE-Stem

2026-04-10 · Kyle Waters, Lucas Nuzzi, Tadhg Looram, Alessandro Tomasiello, Ariel Ghislain Kemogne Kamdoum, Bikun Li, Damien Sileo, Egor Kretov, Francesco Fournier-Facio, Georgios Soloupis, Haile Kassahun, Hew Wolff, Jiaqi Cai, Lianghui Li, Marc Roth, Mohinder Naiya, Naixu Guo, Qicheng Tang, Richard Wheeler, Samuele Sala, Serguei Popov, Steven Dillmann, Yuqi Li arxiv

AI agents hold growing promise for accelerating scientific discovery; yet, a lack of frontier evaluations hinders adoption into real workflows. Expert-written benchmarks have proven effective at measuring AI reasoning, but most at this stage have become saturated and only measure performance on constrained outputs. To help address this gap, we introduce COMPOSITE-STEM, a benchmark of 70 expert-written tasks in physics, biology, chemistry, and mathematics, curated by doctoral-level researchers. Our benchmark combines exact-match grading and criterion-based rubrics with an LLM-as-a-jury grading protocol, allowing more flexible assessment of scientifically meaningful outputs. Using an adapted multimodal Terminus-2 agent harness within the Harbor agentic evaluation framework, we evaluate four frontier models. The top-performing model achieves 21%, demonstrating that COMPOSITE-STEM captures capabilities beyond current agent reach. All tasks are open-sourced with contributor permission to support reproducibility and to promote additional research towards AI's acceleration of scientific progress in these domains.

📄 PDF Abstract BibTeX arXiv:2604.09836

Code (0)

등록된 구현이 없습니다.

CG-GAN: An Interactive Evolutionary GAN-based Approach for Facial Composite Generation)

2019-11-28 · Nicola Zaltron, Luisa Zurlo, Sebastian Risi

Facial composites are graphical representations of an eyewitness's memory of a face. Many digital systems are available for the creation of such composites but are either unable to reproduce features unless previously de…

A Survey of Multimodal Composite Editing and Retrieval

2024-09-09 · Suyan Li, Fuxiang Huang, Lei Zhang

In the real world, where information is abundant and diverse across different modalities, understanding and utilizing various data types to improve retrieval systems is a key focus of research. Multimodal composite retri…

Retrieval Survey

SENTINEL: Taming Uncertainty with Ensemble-based Distributional Reinforcement Learning

2021-02-22 · Hannes Eriksson, Debabrota Basu, Mina Alibeigi, Christos Dimitrakakis

In this paper, we consider risk-sensitive sequential decision-making in Reinforcement Learning (RL). Our contributions are two-fold. First, we introduce a novel and coherent quantification of risk, namely composite risk,…

Decision Making Distributional Reinforcement Learning Model-based Reinforcement Learning reinforcement-learning+3

Neural Network Gaussian Process Considering Input Uncertainty for Composite Structures Assembly

2020-11-21 · Cheolhei Lee, Jianguo Wu, Wenjia Wang, Xiaowei Yue

Developing machine learning enabled smart manufacturing is promising for composite structures assembly process. To improve production quality and efficiency of the assembly process, accurate predictive analysis on dimens…

Stress field prediction in fiber-reinforced composite materials using a deep learning approach

2021-11-01 · Anindya Bhaduri, Ashwini Gupta, Lori Graham-Brady

Computational stress analysis is an important step in the design of material systems. Finite element method (FEM) is a standard approach of performing stress analysis of complex material systems. A way to accelerate stre…