Semantic Segmentation for Compound figures
Scientific literature contains large volumes of unstructured data,with over 30\% of figures constructed as a combination of multiple images, these compound figures cannot be analyzed directly with existing information retrieval tools. In this paper, we propose a semantic segmentation approach for compound figure separation, decomposing the compound figures into "master images". Each master image is one part of a compound figure governed by a subfigure label (typically "(a), (b), (c), etc"). In this way, the separated subfigures can be easily associated with the description information in the caption. In particular, we propose an anchor-based master image detection algorithm, which leverages the correlation between master images and subfigure labels and locates the master images in a two-step manner. First, a subfigure label detector is built to extract the global layout information of the compound figure. Second, the layout information is combined with local features to locate the master images. We validate the effectiveness of proposed method on our labeled testing dataset both quantitatively and qualitatively.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalRetrievalSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
A Data Driven Approach for Compound Figure Separation Using Convolutional Neural Networks
A key problem in automatic analysis and understanding of scientific papers is to extract semantic information from non-textual paper components like figures, diagrams, tables, etc. Much of this work requires a very first…
Transfer LearningA Two-stage Framework for Compound Figure Separation
Scientific literature contains large volumes of complex, unstructured figures that are compound in nature (i.e. composed of multiple images, graphs, and drawings). Separation of these compound figures is critical for inf…
feature selectionInformation RetrievalRetrievalVocal Bursts Valence PredictionAutomatic Separation of Compound Figures in Scientific Articles
Content-based analysis and retrieval of digital images found in scientific articles is often hindered by images consisting of multiple subfigures (compound figures). We address this problem by proposing a method to autom…
ArticlesRetrievalFigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures
Scientific compound figures combine multiple labeled panels into a single image. However, in a PMC-scale crawl of 346,567 compound figures, 16.3% have no caption and 1.8% only have captions shorter than ten words, causin…
Compound Figure Separation of Biomedical Images: Mining Large Datasets for Self-supervised Learning
With the rapid development of self-supervised learning (e.g., contrastive learning), the importance of having large-scale images (even without annotations) for training a more generalizable AI model has been widely recog…
Contrastive LearningImage Augmentationimage-classificationImage Classification+2