CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse paRsing in conversations
Discourse parsing is an important task useful for NLU applications such as summarization, machine comprehension, and emotion recognition. The current discourse parsing datasets based on conversations consists of written English dialogues restricted to a single domain. In this resource paper, we introduce CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse paRsing in conversations. The corpus (code-mixed in Hindi and English) has both audio and transcribed text and is annotated with nine discourse relations. We experiment with various SoTA baseline models; the poor performance of SoTA models highlights the challenges of multi-domain code-mixed corpus, pointing towards the need for developing better models for such realistic settings.
Code (1)
Tasks
Discourse ParsingEmotion RecognitionReading ComprehensionSimilar Papers 제목 키워드 기반
Multimodal Action Quality Assessment
Action quality assessment (AQA) is to assess how well an action is performed. Previous works perform modelling by only the use of visual information, ignoring audio information. We argue that although AQA is highly depen…
Action Quality AssessmentDecoderOptical Flow EstimationMedSumm: A Multimodal Approach to Summarizing Code-Mixed Hindi-English Clinical Queries
In the healthcare domain, summarizing medical questions posed by patients is critical for improving doctor-patient interactions and medical decision-making. Although medical data has grown in complexity and quantity, the…
Decision MakingToxVidLM: A Multimodal Framework for Toxicity Detection in Code-Mixed Videos
In an era of rapidly evolving internet technology, the surge in multimodal content, including videos, has expanded the horizons of online communication. However, the detection of toxic content in this diverse landscape, …
Video ClassificationDiscrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing
Recent work on discrete speech tokenization has paved the way for models that can seamlessly perform multiple tasks across modalities, e.g., speech recognition, text to speech, speech to speech translation. Moreover, lar…
DecoderLanguage ModelingLanguage ModellingLarge Language Model+6Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
We propose a novel approach to train a multi-modal policy from mixed demonstrations without their behavior labels. We develop a method to discover the latent factors of variation in the demonstrations. Specifically, our …
Decoder