Learning Physical Simulation with Message Passing Transformer

2024-06-10 · Zeyi Xu, Yifei Li

Machine learning methods for physical simulation have achieved significant success in recent years. We propose a new universal architecture based on Graph Neural Network, the Message Passing Transformer, which incorporates a Message Passing framework, employs an Encoder-Processor-Decoder structure, and applies Graph Fourier Loss as loss function for model optimization. To take advantage of the past message passing state information, we propose Hadamard-Product Attention to update the node attribute in the Processor, Hadamard-Product Attention is a variant of Dot-Product Attention that focuses on more fine-grained semantics and emphasizes on assigning attention weights over each feature dimension rather than each position in the sequence relative to others. We further introduce Graph Fourier Loss (GFL) to balance high-energy and low-energy components. To improve time performance, we precompute the graph's Laplacian eigenvectors before the training process. Our architecture achieves significant accuracy improvements in long-term rollouts for both Lagrangian and Eulerian dynamical systems over current methods.

📄 PDF Abstract BibTeX arXiv:2406.06060

Code (0)

등록된 구현이 없습니다.

Tasks

AttributeDecoder Graph Neural NetworkModel Optimization

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Adam 설명 없음

Graph Neural Network 설명 없음

MeshGraphNet-Transformer: Scalable Mesh-based Learned Simulation for Solid Mechanics

2026-01-30 · Mikel M. Iparraguirre, Iciar Alfaro, David Gonzalez, Elias Cueto arxiv

We present MeshGraphNet-Transformer (MGN-T), a novel architecture that combines the global modeling capabilities of Transformers with the geometric inductive bias of MeshGraphNets, while preserving a mesh-based graph rep…

EvoMesh: Adaptive Physical Simulation with Hierarchical Graph Evolutions

2024-10-03 · Huayu Deng, Xiangming Zhu, Yunbo Wang, Xiaokang Yang

Graph neural networks have been a powerful tool for mesh-based physical simulation. To efficiently model large-scale systems, existing methods mainly employ hierarchical graph structures to capture multi-scale node relat…

Extended Short- and Long-Range Mesh Learning for Fast and Generalized Garment Simulation

2025-04-16 · Aoran Liu, Kun Hu, Clinton Mo, Changyang Li 외

3D garment simulation is a critical component for producing cloth-based graphics. Recent advancements in graph neural networks (GNNs) offer a promising approach for efficient garment simulation. However, GNNs require ext…

Graph Inductive Biases in Transformers without Message Passing

2023-05-27 · Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soriano 외

Transformers for graph data are increasingly widely studied and successful in numerous learning tasks. Graph inductive biases are crucial for Graph Transformers, and previous works incorporate them using message-passing …

Graph Classification Graph Regression Inductive Bias Node Classification

Expressivity-Preserving GNN Simulation

2023-09-21 · NeurIPS 2023 11

We systematically investigate graph transformations that enable standard message passing to simulate state-of-the-art graph neural networks (GNNs) without loss of expressivity. Using these, many state-of-the-art GNNs can…