Probing the effects of broken symmetries in machine learning
Symmetry is one of the most central concepts in physics, and it is no surprise that it has also been widely adopted as an inductive bias for machine-learning models applied to the physical sciences. This is especially true for models targeting the properties of matter at the atomic scale. Both established and state-of-the-art approaches, with almost no exceptions, are built to be exactly equivariant to translations, permutations, and rotations of the atoms. Incorporating symmetries -- rotations in particular -- constrains the model design space and implies more complicated architectures that are often also computationally demanding. There are indications that non-symmetric models can easily learn symmetries from data, and that doing so can even be beneficial for the accuracy of the model. We put a model that obeys rotational invariance only approximately to the test, in realistic scenarios involving simulations of gas-phase, liquid, and solid water. We focus specifically on physical observables that are likely to be affected -- directly or indirectly -- by symmetry breaking, finding negligible consequences when the model is used in an interpolative, bulk, regime. Even for extrapolative gas-phase predictions, the model remains very stable, even though symmetry artifacts are noticeable. We also discuss strategies that can be used to systematically reduce the magnitude of symmetry breaking when it occurs, and assess their impact on the convergence of observables.
Code (1)
Tasks
Inductive BiasMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Group-Equivariant Autoencoder for Identifying Spontaneously Broken Symmetries
We introduce the group-equivariant autoencoder (GE-autoencoder) -- a deep neural network (DNN) method that locates phase boundaries by determining which symmetries of the Hamiltonian have spontaneously broken at each tem…
Learning Broken Symmetries with Approximate Invariance
Recognizing symmetries in data allows for significant boosts in neural network training, which is especially important where training data are limited. In many cases, however, the exact underlying symmetry is present onl…
Data AugmentationSymmetry's Edge in Cortical Dynamics: Multiscale Dynamics of Ensemble Excitation and Inhibition
Creating a quantitative theory for the cortex presents challenges and raises questions. What are the significant scales--micro, meso, or macroscopic? What are the interactions--pairwise, higher order, or mean-field? And …
Interpretability for Turing Machines
We show that susceptibilities, an interpretability technique developed for neural networks, can identify the presence of algorithmic structure in Turing machines by probing the local loss landscape of a learning problem …
Spontaneous Symmetry Breaking in Neural Networks
We propose a framework to understand the unprecedented performance and robustness of deep neural networks using field theory. Correlations between the weights within the same layer can be described by symmetries in that …