Early Classifying Multimodal Sequences
Often pieces of information are received sequentially over time. When did one collect enough such pieces to classify? Trading wait time for decision certainty leads to early classification problems that have recently gained attention as a means of adapting classification to more dynamic environments. However, so far results have been limited to unimodal sequences. In this pilot study, we expand into early classifying multimodal sequences by combining existing methods. We show our new method yields experimental AUC advantages of up to 8.7%.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationEarly ClassificationSimilar Papers 제목 키워드 기반
Representation Learning of Tangled Key-Value Sequence Data for Early Classification
Key-value sequence data has become ubiquitous and naturally appears in a variety of real-world applications, ranging from the user-product purchasing sequences in e-commerce, to network packet sequences forwarded by rout…
Early ClassificationRepresentation LearningProgressive Modality Reinforcement for Human Multimodal Emotion Recognition From Unaligned Multimodal Sequences
Human multimodal emotion recognition involves time-series data of different modalities, such as natural language, visual motions, and acoustic behaviors. Due to the variable sampling rates for sequences from differen…
Emotion RecognitionMultimodal Emotion RecognitionTime SeriesTime Series AnalysisA Review of Vision-Language Models and their Performance on the Hateful Memes Challenge
Moderation of social media content is currently a highly manual task, yet there is too much content posted daily to do so effectively. With the advent of a number of multimodal models, there is the potential to reduce th…
Diagnosing Alzheimer's Disease using Early-Late Multimodal Data Fusion with Jacobian Maps
Alzheimer's disease (AD) is a prevalent and debilitating neurodegenerative disorder impacting a large aging population. Detecting AD in all its presymptomatic and symptomatic stages is crucial for early intervention and …
Multimodal Foundation Models for Early Disease Detection
Healthcare data now span EHRs, medical imaging, genomics, and wearable sensors, but most diagnostic models still process these modalities in isolation. This limits their ability to capture early, cross-modal disease sign…