Numeracy-600K: Learning Numeracy for Detecting Exaggerated Information in Market Comments
In this paper, we attempt to answer the question of whether neural network models can learn numeracy, which is the ability to predict the magnitude of a numeral at some specific position in a text description. A large benchmark dataset, called Numeracy-600K, is provided for the novel task. We explore several neural network models including CNN, GRU, BiGRU, CRNN, CNN-capsule, GRU-capsule, and BiGRU-capsule in the experiments. The results show that the BiGRU model gets the best micro-averaged F1 score of 80.16{\%}, and the GRU-capsule model gets the best macro-averaged F1 score of 64.71{\%}. Besides discussing the challenges through comprehensive experiments, we also present an important application scenario, i.e., detecting exaggerated information, for the task.
Code (1)
Tasks
PositionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Do NLP Models Know Numbers? Probing Numeracy in Embeddings
The ability to understand and work with numbers (numeracy) is critical for many complex reasoning tasks. Currently, most NLP models treat numbers in text in the same way as other tokens---they embed them as distributed v…
Question AnsweringRevealing the Numeracy Gap: An Empirical Investigation of Text Embedding Models
Text embedding models are widely used in natural language processing applications. However, their capability is often benchmarked on tasks that do not require understanding nuanced numerical information in text. As a res…
Learning Numeracy: A Simple Yet Effective Number Embedding Approach Using Knowledge Graph
Numeracy plays a key role in natural language understanding. However, existing NLP approaches, not only traditional word2vec approach or contextualized transformer-based language models, fail to learn numeracy. As the re…
Graph EmbeddingKnowledge Graph EmbeddingNatural Language UnderstandingInvestigating Numeracy Learning Ability of a Text-to-Text Transfer Model
The transformer-based pre-trained language models have been tremendously successful in most of the conventional NLP tasks. But they often struggle in those tasks where numerical understanding is required. Some possible r…
Transfer LearningRepresenting Numbers in NLP: a Survey and a Vision
NLP systems rarely give special consideration to numbers found in text. This starkly contrasts with the consensus in neuroscience that, in the brain, numbers are represented differently from words. We arrange recent NLP …
Survey