A PAC-Bayesian Generalization Bound for Equivariant Networks
Equivariant networks capture the inductive bias about the symmetry of the learning task by building those symmetries into the model. In this paper, we study how equivariance relates to generalization error utilizing PAC Bayesian analysis for equivariant networks, where the transformation laws of feature spaces are determined by group representations. By using perturbation analysis of equivariant networks in Fourier domain for each layer, we derive norm-based PAC-Bayesian generalization bounds. The bound characterizes the impact of group size, and multiplicity and degree of irreducible representations on the generalization error and thereby provide a guideline for selecting them. In general, the bound indicates that using larger group size in the model improves the generalization error substantiated by extensive numerical experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
Generalization BoundsInductive BiasSimilar Papers 제목 키워드 기반
A PAC-Bayesian approach to generalization for quantum models
Generalization is a central concept in machine learning theory, yet for quantum models, it is predominantly analyzed through uniform bounds that depend on a model's overall capacity rather than the specific function lear…
Quantum Machine LearningGeneralization Bounds for Equivariant Networks on Markov Data
Equivariant neural networks play a pivotal role in analyzing datasets with symmetry properties, particularly in complex data structures. However, integrating equivariance with Markov properties presents notable challenge…
Generalization BoundsImproved Generalization Bounds of Group Invariant / Equivariant Deep Networks via Quotient Feature Spaces
Numerous invariant (or equivariant) neural networks have succeeded in handling invariant data such as point clouds and graphs. However, a generalization theory for the neural networks has not been well developed, because…
Generalization BoundsData Augmentation vs. Equivariant Networks: A Theory of Generalization on Dynamics Forecasting
Exploiting symmetry in dynamical systems is a powerful way to improve the generalization of deep learning. The model learns to be invariant to transformation and hence is more robust to distribution shift. Data augmentat…
Data AugmentationGeneralization BoundsEquivariant score-based generative models provably learn distributions with symmetries efficiently
Symmetry is ubiquitous in many real-world phenomena and tasks, such as physics, images, and molecular simulations. Empirical studies have demonstrated that incorporating symmetries into generative models can provide bett…
Data AugmentationGeneralization BoundsInductive Bias