ChemLabs on ChemO: A Multi-Agent System for Multimodal Reasoning on IChO 2025
Olympiad-level benchmarks in mathematics and physics are crucial testbeds for advanced AI reasoning, but chemistry, with its unique multimodal symbolic language, has remained an open challenge. We introduce ChemO, a new benchmark built from the International Chemistry Olympiad (IChO) 2025. ChemO features two key innovations for automated assessment: Assessment-Equivalent Reformulation (AER), which converts problems requiring visual outputs (e.g., drawing molecules) into computationally tractable formats, and Structured Visual Enhancement (SVE), a diagnostic mechanism to disentangle a model's visual perception capabilities from its core chemical reasoning. To tackle this benchmark, we propose ChemLabs, a hierarchical multi-agent framework that mimics human expert collaboration through specialized agents for problem decomposition, perception, reasoning, and auditing. Experiments on state-of-the-art multimodal models demonstrate that combining SVE with our multi-agent system yields dramatic performance gains. Our top configuration achieves a score of 93.6 out of 100, surpassing an estimated human gold medal threshold and establishing a new state-of-the-art in automated chemical problem-solving. ChemO Dataset: https://huggingface.co/datasets/IDEA-AI4SCI/ChemO
Code (0)
등록된 구현이 없습니다.
Tasks
Multimodal ReasoningSimilar Papers 제목 키워드 기반
Understanding AKT-mediated chemoresistance: the relationship between ion channels and AKT activation
Overcoming chemoresistance is a challenge for multiple chemotherapeutics agents like cisplatin. ABC transporters such as MDR1 or MRPs and PI3K/AKT pathway have been proposed as actors of chemoresistance in several cancer…
Emergence of Chemotactic Strategies with Multi-Agent Reinforcement Learning
Reinforcement learning (RL) is a flexible and efficient method for programming micro-robots in complex environments. Here we investigate whether reinforcement learning can provide insights into biological systems when tr…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Effects of drug resistance in the tumour-immune system with chemotherapy treatment
Cancer is a term used to refer to a large set of diseases. The cancerous cells grow and divide and, as a result, they form tumours that grow in size. The immune system recognise the cancerous cells and attack them, thoug…
Nonlinear cancer chemotherapy: modelling the Norton-Simon hypothesis
A fundamental model of tumor growth in the presence of cytotoxic chemotherapeutic agents is formulated. The model allows to study the role of the Norton-Simon hypothesis in the context of dose-dense chemotherapy. Dose-de…
Evolutionary dynamics in vascularised tumours under chemotherapy
We consider a mathematical model for the evolutionary dynamics of tumour cells in vascularised tumours under chemotherapy. The model comprises a system of coupled partial integro-differential equations for the phenotypic…