paper-with-me

Papers

MolGrapher: Graph-based Visual Recognition of Chemical Structures

2023-08-23 · ICCV 2023 1 · Lucas Morin, Martin Danelljan, Maria Isabel Agea, Ahmed Nassar, Valery Weber, Ingmar Meijer, Peter Staar, Fisher Yu

The automatic analysis of chemical literature has immense potential to accelerate the discovery of new materials and drugs. Much of the critical information in patent documents and scientific articles is contained in figures, depicting the molecule structures. However, automatically parsing the exact chemical structure is a formidable challenge, due to the amount of detailed information, the diversity of drawing styles, and the need for training data. In this work, we introduce MolGrapher to recognize chemical structures visually. First, a deep keypoint detector detects the atoms. Second, we treat all candidate atoms and bonds as nodes and put them in a graph. This construct allows a natural graph representation of the molecule. Last, we classify atom and bond nodes in the graph with a Graph Neural Network. To address the lack of real training data, we propose a synthetic data generation pipeline producing diverse and realistic results. In addition, we introduce a large-scale benchmark of annotated real molecule images, USPTO-30K, to spur research on this critical topic. Extensive experiments on five datasets show that our approach significantly outperforms classical and learning-based methods in most settings. Code, models, and datasets are available.

📄 PDF Abstract BibTeX arXiv:2308.12234

Code (1)

ds4sd/molgrapher 공식 구현 pytorch

Tasks

ArticlesDiversityGraph Neural Network

Methods 이 논문이 사용한 방법론

Graph Neural Network 설명 없음

Similar Papers 제목 키워드 기반

MarkushGrapher: Joint Visual and Textual Recognition of Markush Structures

2025-03-20 · CVPR 2025 1 · Lucas Morin, Valéry Weber, Ahmed Nassar, Gerhard Ingmar Meijer 외

The automated analysis of chemical literature holds promise to accelerate discovery in fields such as material science and drug development. In particular, search capabilities for chemical structures and Markush structur…

Synthetic Data Generation

MarkushGrapher-2: End-to-end Multimodal Recognition of Chemical Structures

2026-03-30 · Tim Strohmeyer, Lucas Morin, Gerhard Ingmar Meijer, Valéry Weber 외 arxiv

Automatically extracting chemical structures from documents is essential for the large-scale analysis of the literature in chemistry. Automatic pipelines have been developed to recognize molecules represented either in f…

GTR-CoT: Graph Traversal as Visual Chain of Thought for Molecular Structure Recognition

2025-06-09 · JingChao Wang, Haote Yang, Jiang Wu, Yifan He 외

Optical Chemical Structure Recognition (OCSR) is crucial for digitizing chemical knowledge by converting molecular images into machine-readable formats. While recent vision-language models (VLMs) have shown potential in …

Image Captioning

MolSight: A Graph-Aware Vision-Language Model for Unified Chemical Image Understanding

2026-07-02 · Wenda Wang, Yihan Tong, Yuwei Hu, Zhewei Wei arxiv

Using molecular large language models (LLMs) as a unified framework for understanding molecular structures and functions is emerging as a new trend in tasks such as molecular design and drug discovery. However, these mod…

Drug Discovery

COMO: Closed-Loop Optical Molecule Recognition with Minimum Risk Training

2026-04-26 · Zhuoqi Lyu, Qing Ke arxiv

Optical chemical structure recognition (OCSR) translates molecular images into machine-readable representations like SMILES strings or molecular graphs, but remains challenging in real-world documents due to inexhaustibl…