paper-with-me

Papers

Adaptive Point Transformer

2024-01-26 · Alessandro Baiocchi, Indro Spinelli, Alessandro Nicolosi, Simone Scardapane

The recent surge in 3D data acquisition has spurred the development of geometric deep learning models for point cloud processing, boosted by the remarkable success of transformers in natural language processing. While point cloud transformers (PTs) have achieved impressive results recently, their quadratic scaling with respect to the point cloud size poses a significant scalability challenge for real-world applications. To address this issue, we propose the Adaptive Point Cloud Transformer (AdaPT), a standard PT model augmented by an adaptive token selection mechanism. AdaPT dynamically reduces the number of tokens during inference, enabling efficient processing of large point clouds. Furthermore, we introduce a budget mechanism to flexibly adjust the computational cost of the model at inference time without the need for retraining or fine-tuning separate models. Our extensive experimental evaluation on point cloud classification tasks demonstrates that AdaPT significantly reduces computational complexity while maintaining competitive accuracy compared to standard PTs. The code for AdaPT is made publicly available.

📄 PDF Abstract BibTeX arXiv:2401.14845

Code (0)

등록된 구현이 없습니다.

Tasks

Point Cloud Classification

Methods 이 논문이 사용한 방법론

Attention 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Multi-Head Attention 설명 없음
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Similar Papers 제목 키워드 기반

Adaptive Channel Encoding Transformer for Point Cloud Analysis

2021-12-05 · Guoquan Xu, Hezhi Cao, Yifan Zhang, Yanxin Ma 외

Transformer plays an increasingly important role in various computer vision areas and remarkable achievements have also been made in point cloud analysis. Since they mainly focus on point-wise transformer, an adaptive ch…

Point Cloud Classification

DAPoinTr: Domain Adaptive Point Transformer for Point Cloud Completion

2024-12-26 · Yinghui Li, Qianyu Zhou, Jingyu Gong, Ye Zhu 외

Point Transformers (PoinTr) have shown great potential in point cloud completion recently. Nevertheless, effective domain adaptation that improves transferability toward target domains remains unexplored. In this paper, …

DecoderDomain AdaptationPoint Cloud CompletionPseudo Label

Adaptive Rank Allocation: Speeding Up Modern Transformers with RaNA Adapters

2025-03-23 · Roberto Garcia, Jerry Liu, Daniel Sorvisto, Sabri Eyuboglu

Large Language Models (LLMs) are computationally intensive, particularly during inference. Neuron-adaptive techniques, which selectively activate neurons in Multi-Layer Perceptron (MLP) layers, offer some speedups but su…

EAGLE: Contextual Point Cloud Generation via Adaptive Continuous Normalizing Flow with Self-Attention

2025-03-05 · Linhao Wang, Qichang Zhang, Yifan Yang, Hao Wang

As 3D point clouds become the prevailing shape representation in computer vision, how to generate high-resolution point clouds has become a pressing issue. Flow-based generative models can effectively perform point cloud…

Point Cloud Generation

SARA-RT: Scaling up Robotics Transformers with Self-Adaptive Robust Attention

2023-12-04 · Isabel Leal, Krzysztof Choromanski, Deepali Jain, Avinava Dubey 외

We present Self-Adaptive Robust Attention for Robotics Transformers (SARA-RT): a new paradigm for addressing the emerging challenge of scaling up Robotics Transformers (RT) for on-robot deployment. SARA-RT relies on the …

Vision-Language-Action