Data Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting
Exploiting symmetry in dynamical systems is a powerful way to improve the generalization of deep learning. The model learns to be invariant to transformation and hence is more robust to distribution shift. Data augmentation and equivariant networks are two major approaches to injecting symmetry into learning. However, their exact role in improving generalization is not well understood. In this work, we derive the generalization bounds for data augmentation and equivariant networks, characterizing their effect on learning in a unified framework. Unlike most prior theories for the i.i.d. setting, we focus on non-stationary dynamics forecasting with complex temporal dependencies.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationGeneralization BoundsSimilar Papers 제목 키워드 기반
Equivariant score-based generative models provably learn distributions with symmetries efficiently
Symmetry is ubiquitous in many real-world phenomena and tasks, such as physics, images, and molecular simulations. Empirical studies have demonstrated that incorporating symmetries into generative models can provide bett…
Data AugmentationGeneralization BoundsInductive BiasEquivariant Data Augmentation for Generalization in Offline Reinforcement Learning
We present a novel approach to address the challenge of generalization in offline reinforcement learning (RL), where the agent learns from a fixed dataset without any additional interaction with the environment. Specific…
Data AugmentationOffline RLreinforcement-learningReinforcement Learning+1Understanding the Role of Equivariance in Self-supervised Learning
Contrastive learning has been a leading paradigm for self-supervised learning, but it is widely observed that it comes at the price of sacrificing useful features (\eg colors) by being invariant to data augmentations. Gi…
Contrastive LearningSelf-Supervised LearningEquivariant Neural Tangent Kernels
Little is known about the training dynamics of equivariant neural networks, in particular how it compares to data augmented training of their non-equivariant counterparts. Recently, neural tangent kernels (NTKs) have eme…
Data Augmentationimage-classificationImage ClassificationMedical Image Analysis+1SE(3)-Equivariant Robot Learning and Control: A Tutorial Survey
Recent advances in deep learning and Transformers have driven major breakthroughs in robotics by employing techniques such as imitation learning, reinforcement learning, and LLM-based multimodal perception and decision-m…
Data AugmentationDeep LearningImitation Learning