Graceful forgetting: Memory as a process
A rational theory of memory is proposed to explain how we can accommodate unbounded sensory input within bounded storage space. Memory is stored as statistics, organized into complex structures that are constantly summarized and compressed to make room for new input. This process, driven by space constraints, is guided by heuristics that optimize the memory for future needs. Sensory input is rapidly encoded as simple statistics that are more slowly elaborated into more abstract constructs. This theory differs from previous accounts of memory by (a) its reliance on statistics, (b) its use of heuristics to guide the choice of statistics, and (c) the emphasis on memory as a process that is intensive, complex, and expensive. The theory is intended as an aid to make sense of our extensive knowledge of memory, and bring us closer to an understanding of memory in functional and mechanistic terms.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Graceful Forgetting in Generative Language Models
Recently, the pretrain-finetune paradigm has become a cornerstone in various deep learning areas. While in general the pre-trained model would promote both effectiveness and efficiency of downstream tasks fine-tuning, st…
The Price of Meaning: Why Every Semantic Memory System Forgets
Every major AI memory system in production today organises information by meaning. That organisation enables generalisation, analogy, and conceptual retrieval -- but it comes at a price. We prove that the same geometric …
Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning
In the scenario of class-incremental learning (CIL), deep neural networks have to adapt their model parameters to non-stationary data distributions, e.g., the emergence of new classes over time. However, CIL models are c…
class-incremental learningClass Incremental LearningDecision MakingIncremental LearningContent Addressable Memory Without Catastrophic Forgetting by Heteroassociation with a Fixed Scaffold
Content-addressable memory (CAM) networks, so-called because stored items can be recalled by partial or corrupted versions of the items, exhibit near-perfect recall of a small number of information-dense patterns below c…
Graceful Forgetting II. Data as a Process
Data are rapidly growing in size and importance for society, a trend motivated by their enabling power. The accumulation of new data, sustained by progress in technology, leads to a boundless expansion of stored data, in…
Dimensionality Reduction