Slot State Space Models
Recent State Space Models (SSMs) such as S4, S5, and Mamba have shown remarkable computational benefits in long-range temporal dependency modeling. However, in many sequence modeling problems, the underlying process is inherently modular and it is of interest to have inductive biases that mimic this modular structure. In this paper, we introduce SlotSSMs, a novel framework for incorporating independent mechanisms into SSMs to preserve or encourage separation of information. Unlike conventional SSMs that maintain a monolithic state vector, SlotSSMs maintains the state as a collection of multiple vectors called slots. Crucially, the state transitions are performed independently per slot with sparse interactions across slots implemented via the bottleneck of self-attention. In experiments, we evaluate our model in object-centric learning, 3D visual reasoning, and long-context video understanding tasks, which involve modeling multiple objects and their long-range temporal dependencies. We find that our proposed design offers substantial performance gains over existing sequence modeling methods. Project page is available at https://slotssms.github.io/
Code (1)
Tasks
MambaState Space ModelsVideo PredictionVideo UnderstandingVisual ReasoningSimilar Papers 제목 키워드 기반
Contrastive Zero-Shot Learning for Cross-Domain Slot Filling with Adversarial Attack
Zero-shot slot filling has widely arisen to cope with data scarcity in target domains. However, previous approaches often ignore constraints between slot value representation and related slot description representation i…
Adversarial Attackslot-fillingSlot FillingZero-Shot Learning+1SlotGAT: Slot-based Message Passing for Heterogeneous Graph Neural Network
Heterogeneous graphs are ubiquitous to model complex data. There are urgent needs on powerful heterogeneous graph neural networks to effectively support important applications. We identify a potential semantic mixing iss…
Graph Neural NetworkHeterogeneous Node ClassificationLink PredictionNode ClassificationLeveraging External Knowledge for Out-Of-Vocabulary Entity Labeling
Dealing with previously unseen slots is a challenging problem in a real-world multi-domain dialogue state tracking task. Other approaches rely on predefined mappings to generate candidate slot keys, as well as their asso…
AttributeDialogue State TrackingMulti-domain Dialogue State TrackingObject-Centric Learning with Slot Mixture Module
Object-centric architectures usually apply a differentiable module to the entire feature map to decompose it into sets of entity representations called slots. Some of these methods structurally resemble clustering algori…
ClusteringObjectProperty PredictionGLASS: Guided Latent Slot Diffusion for Object-Centric Learning
Object-centric learning aims to decompose an input image into a set of meaningful object files (slots). These latent object representations enable a variety of downstream tasks. Yet, object-centric learning struggles…
Conditional Image GenerationImage GenerationObjectObject Discovery