paper-with-me

홈 › Papers

PAINET: A Principled Efficient Transformer for 3D Dynamics Modeling

2025-10-05 · Kai Yang, Yuqi Huang, Junheng Tao, Wanyu Wang, Qitian Wu arxiv

Modeling 3D dynamics is a fundamental problem in multi-body systems across scientific and engineering domains and has important practical implications in object trajectory prediction and simulation. While recent GNN-based approaches have achieved strong performance by enforcing geometric symmetries, encoding high-order features or incorporating neural-ODE mechanics, they typically depend on explicitly observed structures and inherently fail to capture the unobserved interactions that are crucial to complex physical behaviors and dynamics mechanism. In this paper, we propose PAINET, a principled SE(3)-equivariant transformer for learning all-pair interactions in multi-body systems. The model comprises: (1) a novel physics-inspired attention network derived from the minimization trajectory of an energy function, and (2) a parallel decoder that preserves equivariance while enabling efficient inference. Empirical results on diverse real-world benchmarks, including human motion capture, molecular dynamics, and large-scale protein simulations, show that PAINET consistently outperforms recently proposed models, yielding 4.7% to 41.5% error reductions in 3D dynamics prediction with comparable computation costs in terms of time and memory. Our codes, baseline models and datasets are available at https://github.com/Icarus1411/PAINET.

📄 PDF Abstract BibTeX arXiv:2510.04233

Code (0)

등록된 구현이 없습니다.

Tasks

Trajectory Prediction

Similar Papers 제목 키워드 기반

Normalization in Attention Dynamics

2025-10-24 · Nikita Karagodin, Shu Ge, Yury Polyanskiy, Philippe Rigollet arxiv

We study the effect of normalization schemes on token representations in deep transformers. Modeling their evolution as interacting particles on the sphere, we show that normalization acts as a form of speed regulation. …

PDE-SSM: A Spectral State Space Approach to Spatial Mixing in Diffusion Transformers

2026-03-14 · Eshed Gal, Moshe Eliasof, Siddharth Rout, Eldad Haber arxiv

The success of vision transformers-especially for generative modeling-is limited by the quadratic cost and weak spatial inductive bias of self-attention. We propose PDE-SSM, a spatial state-space block that replaces atte…

Stability of Transformers under Layer Normalization

2025-10-10 · Kelvin Kan, Xingjian Li, Benjamin J. Zhang, Tuhin Sahai 외 arxiv

Despite their widespread use, training deep Transformers can be unstable. Layer normalization, a standard component, improves training stability, but its placement has often been ad-hoc. In this paper, we conduct a princ…

Krause Synchronization Transformers

2026-02-12 · Jingkun Liu, Yisong Yue, Max Welling, Yue Song arxiv

Self-attention in Transformers relies on globally normalized softmax weights, causing all tokens to compete for influence at every layer. When composed across depth, this interaction pattern induces strong synchronizatio…

Computational EfficiencyImage Generation

Precision Tracked Transformer via Kalman Filtering, Kriging and Process Noise

2026-05-12 · Bo Long, Deepak Agarwal, Jelena Markovic-Voronov, Yi Wang 외 arxiv

The Transformer is the foundational building block of modern AI, yet offers no principled handling of \emph{uncertainty}, which is prevalent in real applications: cold-start tokens with sparse histories in sequential rec…

Sequential RecommendationQuestion Answering