Permutation-equivariant neural networks applied to dynamics prediction
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries of the image domain. In comparison, there are a number of problems in which there are a number of different inputs which are all 'of the same type' --- multiple particles, multiple agents, multiple stock prices, etc. The corresponding symmetry to this is permutation symmetry, in that the algorithm should not depend on the specific ordering of the input data. We discuss a permutation-invariant neural network layer in analogy to convolutional layers, and show the ability of this architecture to learn to predict the motion of a variable number of interacting hard discs in 2D. In the same way that convolutional layers can generalize to different image sizes, the permutation layer we describe generalizes to different numbers of objects.
Code (2)
Tasks
PredictionTranslationSimilar Papers 제목 키워드 기반
Permutation-Equivariant 2D State Space Models: Theory and Canonical Architecture for Multivariate Time Series
Multivariate time series (MTS) modeling often implicitly imposes an artificial ordering over variables, violating the inherent exchangeability found in many real-world systems where no canonical variable axis exists. We …
Anomaly DetectionPermutation Equivariant Neural Controlled Differential Equations for Dynamic Graph Representation Learning
Dynamic graphs exhibit complex temporal dynamics due to the interplay between evolving node features and changing network structures. Recently, Graph Neural Controlled Differential Equations (Graph Neural CDEs) successfu…
Graph Representation LearningRepresentation LearningThe general theory of permutation equivarant neural networks and higher order graph variational encoders
Previous work on symmetric group equivariant neural networks generally only considered the case where the group acts by permuting the elements of a single vector. In this paper we derive formulae for general permutation …
Graph GenerationGraph LearningLink PredictionMolecular Graph GenerationRelaxing Continuous Constraints of Equivariant Graph Neural Networks for Physical Dynamics Learning
Incorporating Euclidean symmetries (e.g. rotation equivariance) as inductive biases into graph neural networks has improved their generalization ability and data efficiency in unbounded physical dynamics modeling. Howeve…
Graph Neural NetworkSet Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets
Permutation invariant neural networks are a promising tool for making predictions from sets. However, we show that existing permutation invariant architectures, Deep Sets and Set Transformer, can suffer from vanishing or…