paper-with-me

홈 › Papers

Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis

2024-06-12 · Haoxin Liu, Shangqing Xu, Zhiyuan Zhao, Lingkai Kong, Harshavardhan Kamarthi, Aditya B. Sasanur, Megha Sharma, Jiaming Cui, Qingsong Wen, Chao Zhang, B. Aditya Prakash

Time series data are ubiquitous across a wide range of real-world domains. While real-world time series analysis (TSA) requires human experts to integrate numerical series data with multimodal domain-specific knowledge, most existing TSA models rely solely on numerical data, overlooking the significance of information beyond numerical series. This oversight is due to the untapped potential of textual series data and the absence of a comprehensive, high-quality multimodal dataset. To overcome this obstacle, we introduce Time-MMD, the first multi-domain, multimodal time series dataset covering 9 primary data domains. Time-MMD ensures fine-grained modality alignment, eliminates data contamination, and provides high usability. Additionally, we develop MM-TSFlib, the first-cut multimodal time-series forecasting (TSF) library, seamlessly pipelining multimodal TSF evaluations based on Time-MMD for in-depth analyses. Extensive experiments conducted on Time-MMD through MM-TSFlib demonstrate significant performance enhancements by extending unimodal TSF to multimodality, evidenced by over 15% mean squared error reduction in general, and up to 40% in domains with rich textual data. More importantly, our datasets and library revolutionize broader applications, impacts, research topics to advance TSA. The dataset is available at https://github.com/AdityaLab/Time-MMD.

📄 PDF Abstract BibTeX arXiv:2406.08627

Code (2)

adityalab/mm-tsflib 공식 구현 pytorch
adityalab/time-mmd 공식 구현

Tasks

Time SeriesTime Series AnalysisTime Series Forecasting

Methods 이 논문이 사용한 방법론

Library 설명 없음

Similar Papers 제목 키워드 기반

MHMS: Multimodal Hierarchical Multimedia Summarization

2022-04-07 · JieLin Qiu, Jiacheng Zhu, Mengdi Xu, Franck Dernoncourt 외

Multimedia summarization with multimodal output can play an essential role in real-world applications, i.e., automatically generating cover images and titles for news articles or providing introductions to online videos.…

Articles

Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data

2024-11-11 · Kai Kim, Howard Tsai, Rajat Sen, Abhimanyu Das 외

Current forecasting approaches are largely unimodal and ignore the rich textual data that often accompany the time series due to lack of well-curated multimodal benchmark dataset. In this work, we develop TimeText Corpus…

Time Series

MULTISEISMO: A Multimodal Seismic Dataset and Model for Cross-Modal Seismic Understanding

2026-05-25 · Sai Munikoti, Ian Stewart, Chengping Chai, Lisa Linville 외 arxiv

The application of generalist multimodal models (GMMs) to specialized scientific domains remains limited due to the scarcity of comprehensive domain-specific datasets that integrate multiple data modalities beyond text a…

Information RetrievalMultimodal Reasoning

Empowering Time Series Analysis with Large-Scale Multimodal Pretraining

2026-02-05 · Peng Chen, Siyuan Wang, Shiyan Hu, Xingjian Wu 외 arxiv

While existing time series foundation models primarily rely on large-scale unimodal pretraining, they lack complementary modalities to enhance time series understanding. Building multimodal foundation models is a natural…

Time Series ForecastingTime Series AnalysisAnomaly Detection

MemoSen: A Multimodal Dataset for Sentiment Analysis of Memes

2022-06-01 · LREC 2022 6 · Eftekhar Hossain, Omar Sharif, Mohammed Moshiul Hoque

Posting and sharing memes have become a powerful expedient of expressing opinions on social media in recent days. Analysis of sentiment from memes has gained much attention to researchers due to its substantial implicati…

Multimodal Sentiment AnalysisSentiment AnalysisSentiment Classification