Learning Linear Symmetries in Data Using Moment Matching
It is common in machine learning and statistics to use symmetries derived from expert knowledge to simplify problems or improve performance, using methods like data augmentation or penalties. In this paper we consider the unsupervised and semi-supervised problems of learning such symmetries in a distribution directly from data in a model-free fashion. We show that in the worst case this problem is as difficult as the graph automorphism problem. However, if we restrict to the case where the covariance matrix has unique eigenvalues, then the eigenvectors will also be eigenvectors of the symmetry transformation. If we further restrict to finding orthogonal symmetries, then the eigenvalues will be either be 1 or -1, and the problem reduces to determining which eigenvectors are which. We develop and compare theoretically and empirically the effectiveness of different methods of selecting which eigenvectors should have eigenvalue -1 in the symmetry transformation, and discuss how to extend this approach to non-orthogonal cases where we have labels
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationSimilar Papers 제목 키워드 기반
Momentum Conserving Lagrangian Neural Networks
Realistic models of physical world rely on differentiable symmetries that, in turn, correspond to conservation laws. Recent works on Lagrangian and Hamiltonian neural networks show that the underlying symmetries of a sys…
Inductive BiasLagrangian Neural Network with Differentiable Symmetries and Relational Inductive Bias
Realistic models of physical world rely on differentiable symmetries that, in turn, correspond to conservation laws. Recent works on Lagrangian and Hamiltonian neural networks show that the underlying symmetries of a sys…
Inductive BiasNonlinear denoising score matching for enhanced learning of structured distributions
We present a novel method for training score-based generative models which uses nonlinear noising dynamics to improve learning of structured distributions. Generalizing to a nonlinear drift allows for additional structur…
DenoisingImplicit Higher-Order Moment Matching Technique for Model Reduction of Quadratic-bilinear Systems
We propose a projection based multi-moment matching method for model order reduction of quadratic-bilinear systems. The goal is to construct a reduced system that ensures higher-order moment matching for the multivariate…
FormData-Driven Model Reduction by Moment Matching for Linear and Nonlinear Parametric Systems
Theory and methods to obtain parametric reduced-order models by moment matching are presented. The definition of the parametric moment is introduced, and methods (model-based and data-driven) for the approximation of the…