홈 › Papers

Deepchecks: Evaluating Retrieval-Augmented Generation (RAG)

2026-05-14 · Assaf Gerner, Netta Madvil, Nadav Barak, Alex Zaikman, Jonatan Liberman, Liron Hamra, Rotem Brazilay, Shay Tsadok, Yaron Friedman, Neal Harow, Noam Bresler, Shir Chorev, Philip Tannor, Lior Rokach arxiv

Large Language Models (LLMs) augmented with Retrieval-Augmented Generation (RAG) techniques are revolutionizing applications across multiple domains, such as healthcare, finance, and customer service. Despite their potential, evaluating RAG systems remains a complex challenge due to the stochastic nature of generated outputs and the intricate interplay between retrieval and generation components. This paper introduces Deepchecks, a comprehensive framework tailored for evaluating RAG applications. Deepchecks' evaluation framework addresses RAG applications evaluation through a multi-faceted approach, root cause analysis and production monitoring. By ensuring alignment with application-specific requirements, Deepchecks framework provides a robust foundation for assessing reliability, relevance, and user satisfaction in RAG systems.

📄 PDF Abstract BibTeX arXiv:2605.14488

Code (0)

등록된 구현이 없습니다.

Deepchecks: A Library for Testing and Validating Machine Learning Models and Data

2022-03-16 · Shir Chorev, Philip Tannor, Dan Ben Israel, Noam Bressler 외

This paper presents Deepchecks, a Python library for comprehensively validating machine learning models and data. Our goal is to provide an easy-to-use library comprising of many checks related to various types of issues…

BIG-bench Machine Learning

Grounded in Context: Retrieval-Based Method for Hallucination Detection

2025-04-22 · Assaf Gerner, Netta Madvil, Nadav Barak, Alex Zaikman 외

Despite advancements in grounded content generation, production Large Language Models (LLMs) based applications still suffer from hallucinated answers. We present "Grounded in Context" - Deepchecks' hallucination detecti…

Hallucination Natural Language Inference RAG Retrieval

Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts

2026-07-02 · Valentin J. J. Kreileder, Johannes Reisinger, Andreas Fischer arxiv

Retrieval-Augmented Generation (RAG) systems use the question-answering capabilities of Large Language Models (LLMs) to access information outside their parameters. We evaluate if cluster-based semantic chunking improves…

RAD-Bench: Evaluating Large Language Models Capabilities in Retrieval Augmented Dialogues

2024-09-19 · Tzu-Lin Kuo, Feng-Ting Liao, Mu-Wei Hsieh, Fu-Chieh Chang 외

In real-world applications with Large Language Models (LLMs), external retrieval mechanisms - such as Search-Augmented Generation (SAG), tool utilization, and Retrieval-Augmented Generation (RAG) - are often employed to …

RAG Retrieval Retrieval-augmented Generation

mmRAG: A Modular Benchmark for Retrieval-Augmented Generation over Text, Tables, and Knowledge Graphs

2025-05-16 · Chuan Xu, Qiaosheng Chen, Yutong Feng, Gong Cheng

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing the capabilities of large language models. However, existing RAG evaluation predominantly focuses on text retrieval and relies on opaq…

Information Retrieval Knowledge Graphs Question Answering RAG+3