Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages
Microblogging platforms such as Twitter provide active communication channels during mass convergence and emergency events such as earthquakes, typhoons. During the sudden onset of a crisis situation, affected people post useful information on Twitter that can be used for situational awareness and other humanitarian disaster response efforts, if processed timely and effectively. Processing social media information pose multiple challenges such as parsing noisy, brief and informal messages, learning information categories from the incoming stream of messages and classifying them into different classes among others. One of the basic necessities of many of these tasks is the availability of data, in particular human-annotated data. In this paper, we present human-annotated Twitter corpora collected during 19 different crises that took place between 2013 and 2015. To demonstrate the utility of the annotations, we train machine learning classifiers. Moreover, we publish first largest word2vec word embeddings trained on 52 million crisis-related tweets. To deal with tweets language issues, we present human-annotated normalized lexical resources for different lexical variations.
Code (1)
Tasks
Disaster ResponseHumanitarianWord EmbeddingsSimilar Papers 제목 키워드 기반
TweetDIS: A Large Twitter Dataset for Natural Disasters Built using Weak Supervision
Social media is often utilized as a lifeline for communication during natural disasters. Traditionally, natural disaster tweets are filtered from the Twitter stream using the name of the natural disaster and the filtered…
BIG-bench Machine LearningTwitter Job/Employment Corpus: A Dataset of Job-Related Discourse Built with Humans in the Loop
We present the Twitter Job/Employment Corpus, a collection of tweets annotated by a humans-in-the-loop supervised learning framework that integrates crowdsourcing contributions and expertise on the local community and em…
Borrowing or Codeswitching? Annotating for Finer-Grained Distinctions in Language Mixing
We present a new corpus of Twitter data annotated for codeswitching and borrowing between Spanish and English. The corpus contains 9,500 tweets annotated at the token level with codeswitches, borrowings, and named entiti…
TACO -- Twitter Arguments from COnversations
Twitter has emerged as a global hub for engaging in online conversations and as a research corpus for various disciplines that have recognized the significance of its user-generated content. Argument mining is an importa…
Argument MiningCrowdsourcing and annotating NER for Twitter \#drift
We present two new NER datasets for Twitter; a manually annotated set of 1,467 tweets (kappa=0.942) and a set of 2,975 expert-corrected, crowdsourced NER annotated tweets from the dataset described in Finin et al. (2010)…
Named Entity Recognition (NER)NER