CAMformer: Associative Memory is All You Need
Transformers face scalability challenges due to the quadratic cost of attention, which involves dense similarity computations between queries and keys. We propose CAMformer, a novel accelerator that reinterprets attention as an associative memory operation and computes attention scores using a voltage-domain Binary Attention Content Addressable Memory (BA-CAM). This enables constant-time similarity search through analog charge sharing, replacing digital arithmetic with physical similarity sensing. CAMformer integrates hierarchical two-stage top-k filtering, pipelined execution, and high-precision contextualization to achieve both algorithmic accuracy and architectural efficiency. Evaluated on BERT and Vision Transformer workloads, CAMformer achieves over 10x energy efficiency, up to 4x higher throughput, and 6-8x lower area compared to state-of-the-art accelerators--while maintaining near-lossless accuracy.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Associative Adversarial Networks
We propose a higher-level associative memory for learning adversarial networks. Generative adversarial network (GAN) framework has a discriminator and a generator network. The generator (G) maps white noise (z) to data s…
Generative Adversarial NetworkNeural Associative Memory for Dual-Sequence Modeling
Many important NLP problems can be posed as dual-sequence or sequence-to-sequence modeling tasks. Recent advances in building end-to-end neural architectures have been highly successful in solving such tasks. In this wor…
Natural Language InferenceImprove Language Model and Brain Alignment via Associative Memory
Associative memory engages in the integration of relevant information for comprehension in the human cognition system. In this work, we seek to improve alignment between language models and human brain while processing s…
Language ModelingLanguage ModellingEntropic Associative Memory for Manuscript Symbols
Manuscript symbols can be stored, recognized and retrieved from an entropic digital memory that is associative and distributed but yet declarative; memory retrieval is a constructive operation, memory cues to objects not…
RetrievalOscillatory Associative Memory with Exponential Capacity
The slowing of Moore's law and the increasing energy demands of machine learning present critical challenges for both the hardware and machine learning communities, and drive the development of novel computing paradigms.…