Memorization
1개 벤치마크 · 논문 1,088편 · 이 태스크의 논문 보기 →
Benchmarks
Most implemented
mixup: Beyond Empirical Risk Minimization
Wide & Deep Learning for Recommender Systems
Neural Machine Translation in Linear Time
PaLM: Scaling Language Modeling with Pathways
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
Papers
What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests
Large Language Models (LLMs) can memorize and reveal personal information, raising concerns regarding compliance with the EU's GDPR, particularly the Right to Be Forgotten (RTBF). Existing machine unlearning methods assu…
Machine UnlearningMemorizationReasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
The reasoning capabilities of large language models (LLMs) have been a longstanding focus of research. Recent works have further enhanced these capabilities using reinforcement learning (RL), with many new methods claimi…
MathMathematical ReasoningMemorizationReinforcement Learning (RL)Entropy-Memorization Law: Evaluating Memorization Difficulty of Data in LLMs
Large Language Models (LLMs) are known to memorize portions of their training data, sometimes reproducing content verbatim when prompted appropriately. In this work, we investigate a fundamental yet under-explored questi…
MemorizationMMReason: An Open-Ended Multi-Modal Multi-Step Reasoning Benchmark for MLLMs Toward AGI
Reasoning plays a crucial role in advancing Multimodal Large Language Models (MLLMs) toward Artificial General Intelligence. However, existing MLLM benchmarks often fall short in precisely and comprehensively evaluating …
MemorizationListener-Rewarded Thinking in VLMs for Image Preferences
Training robust and generalizable reward models for human visual preferences is essential for aligning text-to-image and text-to-video generative models with human intent. However, current reward models often fail to gen…
MemorizationReinforcement Learning (RL)Where to find Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Test
Grokking, i.e., test performance keeps improving long after training loss converged, has been recently witnessed in neural network training, making the mechanism of generalization and other emerging capabilities such as …
Code GenerationLarge Language ModelMathMemorization