The New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval
Over the past decade, emoji have emerged as a new and widespread form of digital communication, spanning diverse social networks and spoken languages. We propose to treat these ideograms as a new modality in their own right, distinct in their semantic structure from both the text in which they are often embedded as well as the images which they resemble. As a new modality, emoji present rich novel possibilities for representation and interaction. In this paper, we explore the challenges that arise naturally from considering the emoji modality through the lens of multimedia research. Specifically, the ways in which emoji can be related to other common modalities such as text and images. To do so, we first present a large scale dataset of real-world emoji usage collected from Twitter. This dataset contains examples of both text-emoji and image-emoji relationships. We present baseline results on the challenge of predicting emoji from both text and images, using state-of-the-art neural networks. Further, we offer a first consideration into the problem of how to account for new, unseen emoji - a relevant issue as the emoji vocabulary continues to expand on a yearly basis. Finally, we present results for multimedia retrieval using emoji as queries.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalSimilar Papers 제목 키워드 기반
Predict Emoji Combination with Retrieval Strategy
As emojis are widely used in social media, people not only use an emoji to express their emotions or mention things but also extend its usage to represent complicate emotions, concepts or activities by combining multiple…
RetrievalA Federated Approach to Predict Emojis in Hindi Tweets
The use of emojis provide for adding a visual modality to textual communication.The task of predicting emojis however provides a challenge for computational approaches as emoji use tends to cluster into the frequently us…
Federated LearningPrivacy PreservingA Federated Approach to Predicting Emojis in Hindi Tweets
The use of emojis affords a visual modality to, often private, textual communication. The task of predicting emojis however provides a challenge for machine learning as emoji use tends to cluster into the frequently used…
Federated LearningEmoji Retrieval from Gibberish or Garbled Social Media Text: A Novel Methodology and A Case Study
Emojis are widely used across social media platforms but are often lost in noisy or garbled text, posing challenges for data analysis and machine learning. Conventional preprocessing approaches recommend removing such te…
Smile, Be Happy :) Emoji Embedding for Visual Sentiment Analysis
Due to the lack of large-scale datasets, the prevailing approach in visual sentiment analysis is to leverage models trained for object classification in large datasets like ImageNet. However, objects are sentiment neutra…
Sentiment AnalysisTransfer Learning