Partial Channel Dependence with Channel Masks for Time Series Foundation Models
Recent advancements in foundation models have been successfully extended to the time series (TS) domain, facilitated by the emergence of large-scale TS datasets. However, previous efforts have primarily focused on designing model architectures to address explicit heterogeneity among datasets such as various numbers of channels, while often overlooking implicit heterogeneity such as varying dependencies between channels. In this work, we introduce the concept of partial channel dependence (PCD), which enables a more sophisticated adjustment of channel dependencies based on dataset-specific information. To achieve PCD, we propose a channel mask that captures the relationships between channels within a dataset using two key components: 1) a correlation matrix that encodes relative dependencies between channels, and 2) domain parameters that learn the absolute dependencies specific to each dataset, refining the correlation matrix. We validate the effectiveness of PCD across four tasks in TS including forecasting, classification, imputation, and anomaly detection, under diverse settings, including few-shot and zero-shot scenarios with both TS foundation models and single-task models. Code is available at https://github.com/seunghan96/CM.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionImputationTime SeriesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Electrodynamics Correlates Knock-on and Knock-off: Current is Spatially Uniform in Ion Channels
Ions in channels have been imagined as hard balls in a macroscopic mechanical model, for a very long time. Hard balls interact by collisions in such models, randomly knocking each other on and off `binding' sites in ther…
MCformer: Multivariate Time Series Forecasting with Mixed-Channels Transformer
The massive generation of time-series data by largescale Internet of Things (IoT) devices necessitates the exploration of more effective models for multivariate time-series forecasting. In previous models, there was a pr…
Multivariate Time Series ForecastingTime SeriesTime Series ForecastingEnhancement of Spatial Clustering-Based Time-Frequency Masks using LSTM Neural Networks
Recent works have shown that Deep Recurrent Neural Networks using the LSTM architecture can achieve strong single-channel speech enhancement by estimating time-frequency masks. However, these models do not naturally gene…
ClusteringSpeech EnhancementDual Channel Multi-Attention in ViT for Biometric Authentication using Forehead Subcutaneous Vein Pattern and Periocular Pattern
Traditional biometric systems, like face and fingerprint recognition, have encountered significant setbacks due to wearing face masks and hygiene concerns. To meet the challenges of the partially covered face due to face…
Channel-Aware Low-Rank Adaptation in Time Series Forecasting
The balance between model capacity and generalization has been a key focus of recent discussions in long-term time series forecasting. Two representative channel strategies are closely associated with model expressivity …
Time SeriesTime Series Forecasting