Conditional COT-GAN for Video Prediction with Kernel Smoothing
Causal Optimal Transport (COT) results from imposing a temporal causality constraint on classic optimal transport problems, which naturally generates a new concept of distances between distributions on path spaces. The first application of the COT theory for sequential learning was given in Xu et al. (2020), where COT-GAN was introduced as an adversarial algorithm to train implicit generative models optimized for producing sequential data. Relying on (Xu et al., 2020), the contribution of the present paper is twofold. First, we develop a conditional version of COT-GAN suitable for sequence prediction. This means that the dataset is now used in order to learn how a sequence will evolve given the observation of its past evolution. Second, we improve on the convergence results by working with modifications of the empirical measures via kernel smoothing due to (Pflug and Pichler (2016)). The resulting kernel conditional COT-GAN algorithm is illustrated with an application for video prediction.
Code (1)
Tasks
PredictionQuantizationVideo PredictionSimilar Papers 제목 키워드 기반
Real-time Online Video Detection with Temporal Smoothing Transformers
Streaming video recognition reasons about objects and their actions in every frame of a video. A good streaming recognition model captures both long-term dynamics and short-term changes of video. Unfortunately, in most e…
Action AnticipationAction DetectionOnline Action DetectionVideo RecognitionPAS: A Training-Free Stabilizer for Temporal Encoding in Video LLMs
Video LLMs suffer from temporal inconsistency: small shifts in frame timing can flip attention and suppress relevant frames. We trace this instability to the common extension of Rotary Position Embeddings to video throug…
Smoothing Deep Reinforcement Learning for Power Control for Spectrum Sharing in Cognitive Radios
The spectrum sharing in a cognitive radio system is related with a secondary user sharing common spectrum with a primary user for power transmit without induc- ing harmful inference. The deep reinforcement learning has b…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Non-Asymptotic Error Bounds for SMC with Biased Proposals: Application to Conditional Diffusion Sampling
Sequential Monte Carlo (SMC) methods are a natural tool for post-hoc conditioning of pretrained generative models, but in many applications the mutation kernels used by the particle system are biased approximations of an…
Measuring Differences between Conditional Distributions using Kernel Embeddings
Comparing conditional distributions is a fundamental challenge in statistics and machine learning, with applications across a wide range of domains. While proposed methods for measuring discrepancies using kernel embeddi…