Titans Revisited: A Lightweight Reimplementation and Critical Analysis of a Test-Time Memory Model
By the end of 2024, Google researchers introduced Titans: Learning at Test Time, a neural memory model achieving strong empirical results across multiple tasks. However, the lack of publicly available code and ambiguities in the original description hinder reproducibility. In this work, we present a lightweight reimplementation of Titans and conduct a comprehensive evaluation on Masked Language Modeling, Time Series Forecasting, and Recommendation tasks. Our results reveal that Titans does not always outperform established baselines due to chunking. However, its Neural Memory component consistently improves performance compared to attention-only models. These findings confirm the model's innovative potential while highlighting its practical limitations and raising questions for future research.
Code (0)
등록된 구현이 없습니다.
Tasks
Time Series ForecastingSimilar Papers 제목 키워드 기반
Towards Remote Sensing Change Detection with Neural Memory
Remote sensing change detection is essential for environmental monitoring, urban planning, and related applications. However, current methods often struggle to capture long-range dependencies while maintaining computatio…
Computational EfficiencyChange DetectionAgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
As Large Language Models (LLMs) evolve into autonomous agents, the need for unified evaluation infrastructure becomes critical. However, current evaluation pipelines remain highly fragmented and tightly coupled, hinderin…
Deep-Learning Ensembles for Skin-Lesion Segmentation, Analysis, Classification: RECOD Titans at ISIC Challenge 2018
This extended abstract describes the participation of RECOD Titans in parts 1 to 3 of the ISIC Challenge 2018 "Skin Lesion Analysis Towards Melanoma Detection" (MICCAI 2018). Although our team has a long experience with …
AttributeClassificationGeneral ClassificationLesion Segmentation+2LANTERN: Layered Archival and Temporal Episodic Retrieval Network for Long-Context LLM Conversations
Large language models discard critical details when conversation history is compacted to fit within finite context windows. We present LANTERN (Layered Archival aNd Temporal Episodic Retrieval Network), a lightweight mem…
Honk: A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting
We describe Honk, an open-source PyTorch reimplementation of convolutional neural networks for keyword spotting that are included as examples in TensorFlow. These models are useful for recognizing "command triggers" in s…
Keyword Spottingspeech-recognitionSpeech Recognition