Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physics
This study introduces a novel transformer model optimized for large-scale point cloud processing in scientific domains such as high-energy physics (HEP) and astrophysics. Addressing the limitations of graph neural networks and standard transformers, our model integrates local inductive bias and achieves near-linear complexity with hardware-friendly regular operations. One contribution of this work is the quantitative analysis of the error-complexity tradeoff of various sparsification techniques for building efficient transformers. Our findings highlight the superiority of using locality-sensitive hashing (LSH), especially OR & AND-construction LSH, in kernel approximation for large-scale point cloud data with local inductive bias. Based on this finding, we propose LSH-based Efficient Point Transformer (HEPT), which combines E$^2$LSH with OR & AND constructions and is built upon regular computations. HEPT demonstrates remarkable performance on two critical yet time-consuming HEP tasks, significantly outperforming existing GNNs and transformers in accuracy and computational speed, marking a significant advancement in geometric deep learning and large-scale scientific data processing. Our code is available at https://github.com/Graph-COM/HEPT.
Code (1)
Tasks
Inductive BiasMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Application of Structured State Space Models to High energy physics with locality-sensitive hashing
Modern high-energy physics (HEP) experiments are increasingly challenged by the vast size and complexity of their datasets, particularly regarding large-scale point cloud processing and long sequences. In this study, to …
Inductive BiasMambaState Space ModelsEfficient Bitmap-based Indexing and Retrieval of Similarity Search Image Queries
Finding similar images is a necessary operation in many multimedia applications. Images are often represented and stored as a set of high-dimensional features, which are extracted using localized feature extraction algor…
RetrievalExperimental Analysis of Machine Learning Techniques for Finding Search Radius in Locality Sensitive Hashing
Finding similar data in high-dimensional spaces is one of the important tasks in multimedia applications. Approaches introduced to find exact searching techniques often use tree-based index structures which are known to …
LAHNet: Local Attentive Hashing Network for Point Cloud Registration
Most existing learning-based point cloud descriptors for point cloud registration focus on perceiving local information of point clouds to generate distinctive features. However, a reasonable and broader receptive field …
Point Cloud RegistrationPoint CloudsFalconn++: A Locality-sensitive Filtering Approach for Approximate Nearest Neighbor Search
We present Falconn++, a novel locality-sensitive filtering approach for approximate nearest neighbor search on angular distance. Falconn++ can filter out potential far away points in any hash bucket \textit{before} query…