Exploiting Inductive Biases in Video Modeling through Neural CDEs
We introduce a novel approach to video modeling that leverages controlled differential equations (CDEs) to address key challenges in video tasks, notably video interpolation and mask propagation. We apply CDEs at varying resolutions leading to a continuous-time U-Net architecture. Unlike traditional methods, our approach does not require explicit optical flow learning, and instead makes use of the inherent continuous-time features of CDEs to produce a highly expressive video model. We demonstrate competitive performance against state-of-the-art models for video interpolation and mask propagation tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networks
Predicting future video frames is extremely challenging, as there are many factors of variation that make up the dynamics of how frames change through time. Previously proposed solutions require complex inductive biases …
Inductive BiasOptical Flow EstimationVideo PredictionVocal Bursts Intensity PredictionUniversal linguistic inductive biases via meta-learning
How do learners acquire languages from the limited data available to them? This process must involve some inductive biases - factors that affect how a learner generalizes - but it is unclear which inductive biases can ex…
Language AcquisitionMeta-LearningVideo Transformers: A Survey
Transformer models have shown great success handling long-range interactions, making them a promising tool for modeling video. However, they lack inductive biases and scale quadratically with input length. These limitati…
Action ClassificationSelf-Supervised LearningSurveyCounterfactual Generative Networks
Neural networks are prone to learning shortcuts -- they often model simple correlations, ignoring more complex ones that potentially generalize better. Prior works on image classification show that instead of learning a …
ClassificationcounterfactualGeneral ClassificationGPU+4Transferring Inductive Biases through Knowledge Distillation
Having the right inductive biases can be crucial in many tasks or scenarios where data or computing resources are a limiting factor, or where training data is not perfectly representative of the conditions at test time. …
Knowledge Distillation