Temporal Weights
In artificial neural networks, weights are a static representation of synapses. However, synapses are not static, they have their own interacting dynamics over time. To instill weights with interacting dynamics, we use a model describing synchronization that is capable of capturing core mechanisms of a range of neural and general biological phenomena over time. An ideal fit for these Temporal Weights (TW) are Neural ODEs, with continuous dynamics and a dependency on time. The resulting recurrent neural networks efficiently model temporal dynamics by computing on the ordering of sequences, and the length and scale of time. By adding temporal weights to a model, we demonstrate better performance, smaller models, and data efficiency on sparse, irregularly sampled time series datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
Time SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Initialization Strategies of Spatio-Temporal Convolutional Neural Networks
We propose a new way of incorporating temporal information present in videos into Spatial Convolutional Neural Networks (ConvNets) trained on images, that avoids training Spatio-Temporal ConvNets from scratch. We describ…
Dynamic Weight Alignment for Temporal Convolutional Neural Networks
In this paper, we propose a method of improving temporal Convolutional Neural Networks (CNN) by determining the optimal alignment of weights and inputs using dynamic programming. Conventional CNN convolutions linearly ma…
Dynamic Time WarpingTime Series AnalysisTEAGS: Time-aware Text Embedding Approach to Generate Subgraphs
Contagions (e.g. virus, gossip) spread over the nodes in propagation graphs. We can use the temporal and textual data of the nodes to compute the edge weights and then generate subgraphs with highly relevant nodes. This …
Learning Optimal Interaction Weights in Multi-Agents Systems
This paper presents a spatio-temporal inverse optimal control framework for understanding interactions in multi-agent systems (MAS). We employ a graph representation approach and model the dynamics of interactions betwee…
Deep Adaptive Temporal Pooling for Activity Recognition
Deep neural networks have recently achieved competitive accuracy for human activity recognition. However, there is room for improvement, especially in modeling long-term temporal importance and determining the activity r…
Activity RecognitionHuman Activity Recognition