Star with Bilinear Mapping
Contextual modeling is crucial for robust visual representation learning, especially in computer vision. Although Transformers have become a leading architecture for vision tasks due to their attention mechanism, the quadratic complexity of full attention operations presents substantial computational challenges. To address this, we introduce Star with Bilinear Mapping (SBM), a Transformer-like architecture that achieves global contextual modeling with linear complexity. SBM employs a bilinear mapping module (BM) with low-rank decomposition strategy and star operations (element-wise multiplication) to efficiently capture global contextual information. Our model demonstrates competitive performance on image classification and semantic segmentation tasks, delivering significant computational efficiency gains compared to traditional attention-based models.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational Efficiencyimage-classificationImage ClassificationRepresentation LearningSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
STaR: Knowledge Graph Embedding by Scaling, Translation and Rotation
The bilinear method is mainstream in Knowledge Graph Embedding (KGE), aiming to learn low-dimensional representations for entities and relations in Knowledge Graph (KG) and complete missing links. Most of the existing wo…
Graph EmbeddingKnowledge Graph EmbeddingLink PredictionTranslationSTaR: Knowledge Graph Embedding by Scaling, Translation and Rotation
The bilinear method is mainstream in Knowledge Graph Embedding (KGE), aiming to learn low-dimensional representations for entities and relations in Knowledge Graph (KG) and complete missing links. Most of the existing wo…
Graph EmbeddingKnowledge Graph EmbeddingLink PredictionTranslationBest Arm Identification in Graphical Bilinear Bandits
We introduce a new graphical bilinear bandit problem where a learner (or a \emph{central entity}) allocates arms to the nodes of a graph and observes for each edge a noisy bilinear reward representing the interaction bet…
Learning to Negate Adjectives with Bilinear Models
We learn a mapping that negates adjectives by predicting an adjective{'}s antonym in an arbitrary word embedding model. We show that both linear models and neural networks improve on this task when they have access to a …
Word EmbeddingsOn the Convergence of Stochastic Extragradient for Bilinear Games using Restarted Iteration Averaging
We study the stochastic bilinear minimax optimization problem, presenting an analysis of the same-sample Stochastic ExtraGradient (SEG) method with constant step size, and presenting variations of the method that yield f…