Learnt dynamics generalizes across tasks, datasets, and populations
Differentiating multivariate dynamic signals is a difficult learning problem as the feature space may be large yet often only a few training examples are available. Traditional approaches to this problem either proceed from handcrafted features or require large datasets to combat the m >> n problem. In this paper, we show that the source of the problem---signal dynamics---can be used to our advantage and noticeably improve classification performance on a range of discrimination tasks when training data is scarce. We demonstrate that self-supervised pre-training guided by signal dynamics produces embedding that generalizes across tasks, datasets, data collection sites, and data distributions. We perform an extensive evaluation of this approach on a range of tasks including simulated data, keyword detection problem, and a range of functional neuroimaging data, where we show that a single embedding learnt on healthy subjects generalizes across a number of disorders, age groups, and datasets.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Training Dynamics for Text Summarization Models
Pre-trained language models (e.g. BART) have shown impressive results when fine-tuned on large summarization datasets. However, little is understood about this fine-tuning process, including what knowledge is retained fr…
HallucinationNews SummarizationText SummarizationTraining Dynamics for Text Summarization Models
Pre-trained language models (e.g. BART) have shown impressive results when fine-tuned on large summarization datasets. However, little is understood about this fine-tuning process, including what knowledge is retained fr…
HallucinationNews SummarizationText Summarization3D-OES: Viewpoint-Invariant Object-Factorized Environment Simulators
We propose an action-conditioned dynamics model that predicts scene changes caused by object and agent interactions in a viewpoint-invariant 3D neural scene representation space, inferred from RGB-D videos. In this 3D fe…
Graph Neural NetworkObjectLearning to Optimize with Dynamic Mode Decomposition
Designing faster optimization algorithms is of ever-growing interest. In recent years, learning to learn methods that learn how to optimize demonstrated very encouraging results. Current approaches usually do not effecti…
SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision
A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations, like images, using priors informed by Hamiltonian mechanics. While these models have important potential applicati…
Autonomous DrivingImage Reconstruction