paper-with-me

Papers

Transformer with Implicit Edges for Particle-based Physics Simulation

2022-07-22 · Yidi Shao, Chen Change Loy, Bo Dai

Particle-based systems provide a flexible and unified way to simulate physics systems with complex dynamics. Most existing data-driven simulators for particle-based systems adopt graph neural networks (GNNs) as their network backbones, as particles and their interactions can be naturally represented by graph nodes and graph edges. However, while particle-based systems usually contain hundreds even thousands of particles, the explicit modeling of particle interactions as graph edges inevitably leads to a significant computational overhead, due to the increased number of particle interactions. Consequently, in this paper we propose a novel Transformer-based method, dubbed as Transformer with Implicit Edges (TIE), to capture the rich semantics of particle interactions in an edge-free manner. The core idea of TIE is to decentralize the computation involving pair-wise particle interactions into per-particle updates. This is achieved by adjusting the self-attention module to resemble the update formula of graph edges in GNN. To improve the generalization ability of TIE, we further amend TIE with learnable material-specific abstract particles to disentangle global material-wise semantics from local particle-wise semantics. We evaluate our model on diverse domains of varying complexity and materials. Compared with existing GNN-based methods, without bells and whistles, TIE achieves superior performance and generalization across all these domains. Codes and models are available at https://github.com/ftbabi/TIE_ECCV2022.git.

📄 PDF Abstract BibTeX arXiv:2207.10860

Code (1)

ftbabi/tie_eccv2022 공식 구현 pytorch

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Multi-Head Attention 설명 없음
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Position-Wise Feed-Forward Layer 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Similar Papers 제목 키워드 기반

GNS: A generalizable Graph Neural Network-based simulator for particulate and fluid modeling

2022-11-18 · Krishna Kumar, Joseph Vantassel

We develop a PyTorch-based Graph Network Simulator (GNS) that learns physics and predicts the flow behavior of particulate and fluid systems. GNS discretizes the domain with nodes representing a collection of material po…

Graph Neural Network

Implicit Quantile Neural Networks for Jet Simulation and Correction

2021-11-22 · Braden Kronheim, Michelle P. Kuchera, Harrison B. Prosper, Raghuram Ramanujan

Reliable modeling of conditional densities is important for quantitative scientific fields such as particle physics. In domains outside physics, implicit quantile neural networks (IQN) have been shown to provide accurate…

SiT: Simulation Transformer for Particle-based Physics Simulation

2021-09-29 · Yidi Shao, Chen Change Loy, Bo Dai

Most existing particle-based simulators adopt graph convolutional networks (GCNs) to model the underlying physics of particles. However, they force particles to interact with all neighbors without selection, and they fa…

Learning the Physics of Particle Transport via Transformers

2021-09-08 · Oscar Pastor-Serrano, Zoltán Perkó

Particle physics simulations are the cornerstone of nuclear engineering applications. Among them radiotherapy (RT) is crucial for society, with 50% of cancer patients receiving radiation treatments. For the most precise …

Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators

2024-02-19 · Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber 외

Neural operators, serving as physics surrogate models, have recently gained increased interest. With ever increasing problem complexity, the natural question arises: what is an efficient way to scale neural operators to …