Synthetically Trained Icon Proposals for Parsing and Summarizing Infographics
Widely used in news, business, and educational media, infographics are
handcrafted to effectively communicate messages about complex and often
abstract topics including ways to conserve the environment' and understanding
the financial crisis'. Composed of stylistically and semantically diverse
visual and textual elements, infographics pose new challenges for computer
vision. While automatic text extraction works well on infographics, computer
vision approaches trained on natural images fail to identify the stand-alone
visual elements in infographics, or `icons'. To bridge this representation gap,
we propose a synthetic data generation strategy: we augment background patches
in infographics from our Visually29K dataset with Internet-scraped icons which
we use as training data for an icon proposal mechanism. On a test set of 1K
annotated infographics, icons are located with 38% precision and 34% recall
(the best model trained with natural images achieves 14% precision and 7%
recall). Combining our icon proposals with icon classification and text
extraction, we present a multi-modal summarization application. Our application
takes an infographic as input and automatically produces text tags and visual
hashtags that are textually and visually representative of the infographic's
topics respectively.
Code (1)
Tasks
Synthetic Data GenerationSimilar Papers 제목 키워드 기반
Lexicon-injected Semantic Parsing for Task-Oriented Dialog
Recently, semantic parsing using hierarchical representations for dialog systems has captured substantial attention. Task-Oriented Parse (TOP), a tree representation with intents and slots as labels of nested tree nodes,…
Semantic ParsingNon-Deterministic Segmentation for Chinese Lattice Parsing
Parsing Chinese critically depends on correct word segmentation for the parser since incorrect segmentation inevitably causes incorrect parses. We investigate a pipeline approach to segmentation and parsing using word la…
Morphological AnalysisSegmentationSentenceSynthetic vs. Real Reference Strings for Citation Parsing, and the Importance of Re-training and Out-Of-Sample Data for Meaningful Evaluations: Experiments with GROBID, GIANT and Cora
Citation parsing, particularly with deep neural networks, suffers from a lack of training data as available datasets typically contain only a few thousand training instances. Manually labelling citation strings is very t…
Semantic Parsing with Semi-Supervised Sequential Autoencoders
We present a novel semi-supervised approach for sequence transduction and apply it to semantic parsing. The unsupervised component is based on a generative model in which latent sentences generate the unpaired logical fo…
Semantic ParsingOn Eliciting Syntax from Language Models via Hashing
Unsupervised parsing, also known as grammar induction, aims to infer syntactic structure from raw text. Recently, binary representation has exhibited remarkable information-preserving capabilities at both lexicon and syn…
Constituency Grammar Induction