ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsing
Most molecular diagram parsers recover chemical structure from raster images (e.g., PNGs). However, many PDFs include commands giving explicit locations and shapes for characters, lines, and polygons. We present a new parser that uses these born-digital PDF primitives as input. The parsing model is fast and accurate, and does not require GPUs, Optical Character Recognition (OCR), or vectorization. We use the parser to annotate raster images and then train a new multi-task neural network for recognizing molecules in raster images. We evaluate our parsers using SMILES and standard benchmarks, along with a novel evaluation protocol comparing molecular graphs directly that supports automatic error compilation and reveals errors missed by SMILES-based evaluation. On the synthetic USPTO benchmark, our born-digital parser obtains a recognition rate of 98.4% (1% higher than previous models) and our relatively simple neural parser for raster images obtains a rate of 85% using less training data than existing neural approaches (thousands vs. millions of molecules).
Code (2)
Tasks
Optical Character RecognitionOptical Character Recognition (OCR)Similar Papers 제목 키워드 기반
Mol-Instructions: A Large-Scale Biomolecular Instruction Dataset for Large Language Models
Large Language Models (LLMs), with their remarkable task-handling capabilities and innovative outputs, have catalyzed significant advancements across a spectrum of fields. However, their proficiency within specialized do…
Catalytic activity predictionChemical-Disease Interaction ExtractionChemical Entity RecognitionChemical-Protein Interaction Extraction+12DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams
Diagram question answering (DQA) requires models to interpret structured visual representations such as charts, maps, infographics, circuit schematics, and scientific diagrams. Recent vision-language models (VLMs) often …
Question AnsweringVisual ReasoningFrom Text to Visuals: Using LLMs to Generate Math Diagrams with Vector Graphics
Advances in large language models (LLMs) offer new possibilities for enhancing math education by automating support for both teachers and students. While prior work has focused on generating math problems and high-qualit…
MathQuestion AnsweringVector GraphicsVisual Question AnsweringMultimodal Search in Chemical Documents and Reactions
We present a multimodal search tool that facilitates retrieval of chemical reactions, molecular structures, and associated text from scientific literature. Queries may combine molecular diagrams, textual descriptions, an…
RetrievalStarVector: Generating Scalable Vector Graphics Code from Images and Text
Scalable Vector Graphics (SVGs) are vital for modern image rendering due to their scalability and versatility. Previous SVG generation methods have focused on curve-based vectorization, lacking semantic understanding, of…
Code GenerationLanguage ModelingLanguage ModellingLarge Language Model+2