Retrieval Quality at Context Limit
The ability of large language models (LLMs) to recall and retrieve information from long contexts is critical for many real-world applications. Prior work (Liu et al., 2023) reported that LLMs suffer significant drops in retrieval accuracy for facts placed in the middle of large contexts, an effect known as "Lost in the Middle" (LITM). We find the model Gemini 2.5 Flash can answer needle-in-a-haystack questions with great accuracy regardless of document position including when the document is nearly at the input context limit. Our results suggest that the "Lost in the Middle" effect is not present for simple factoid Q\&A in Gemini 2.5 Flash, indicating substantial improvements in long-context retrieval.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Context Quality Matters in Training Fusion-in-Decoder for Extractive Open-Domain Question Answering
Retrieval-augmented generation models augment knowledge encoded in a language model by providing additional relevant external knowledge (context) during generation. Although it has been shown that the quantity and qualit…
DecoderLanguage ModelingLanguage ModellingOpen-Domain Question Answering+3Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddings
A limitation of modern document retrieval embedding methods is that they typically encode passages (chunks) from the same documents independently, often overlooking crucial contextual information from the rest of the doc…
ChunkingComputational EfficiencyRepresentation LearningRetrievalCan Small Language Models Use What They Retrieve? An Empirical Study of Retrieval Utilization Across Model Scale
Retrieval augmented generation RAG is widely deployed to improve factual accuracy in language models yet it remains unclear whether smaller models of size 7B parameters or less can effectively utilize retrieved informati…
Context Selection for Hypothesis and Statistical Evidence Extraction from Full-Text Scientific Articles
Extracting hypotheses and their supporting statistical evidence from full-text scientific articles is central to the synthesis of empirical findings, but remains difficult due to document length and the distribution of s…
LoVR: A Benchmark for Long Video Retrieval in Multimodal Contexts
Long videos contain a vast amount of information, making video-text retrieval an essential and challenging task in multimodal learning. However, existing benchmarks suffer from limited video duration, low-quality caption…
Caption GenerationRetrievalText RetrievalVideo Retrieval+2