Evaluating Search Engines and Large Language Models for Answering Health Questions
Search engines (SEs) have traditionally been primary tools for information seeking, but the new Large Language Models (LLMs) are emerging as powerful alternatives, particularly for question-answering tasks. This study compares the performance of four popular SEs, seven LLMs, and retrieval-augmented (RAG) variants in answering 150 health-related questions from the TREC Health Misinformation (HM) Track. Results reveal SEs correctly answer between 50 and 70% of questions, often hindered by many retrieval results not responding to the health question. LLMs deliver higher accuracy, correctly answering about 80% of questions, though their performance is sensitive to input prompts. RAG methods significantly enhance smaller LLMs' effectiveness, improving accuracy by up to 30% by integrating retrieval evidence.
Code (1)
Tasks
MisinformationNavigateQuestion AnsweringRAGRetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Language Models as Context-sensitive Word Search Engines
Context-sensitive word search engines are writing assistants that support word choice, phrasing, and idiomatic language use by indexing large-scale n-gram collections and implementing a wildcard search. However, search r…
Language ModelingLanguage ModellingLarge Language Models, Knowledge Graphs and Search Engines: A Crossroads for Answering Users' Questions
Much has been discussed about how Large Language Models, Knowledge Graphs and Search Engines can be combined in a synergistic manner. A dimension largely absent from current academic discourse is the user perspective. In…
Knowledge GraphsSearchRAG: Can Search Engines Be Helpful for LLM-based Medical Question Answering?
Large Language Models (LLMs) have shown remarkable capabilities in general domains but often struggle with tasks requiring specialized knowledge. Conventional Retrieval-Augmented Generation (RAG) techniques typically ret…
Medical Question AnsweringQuestion AnsweringRAGRetrieval-augmented GenerationAnswer Extraction for Why Arabic Questions Answering Systems: EWAQ
With the increasing amount of web information, questions answering systems becomes very important to allow users to access to direct answers for their requests. This paper presents an Arabic Questions Answering Systems b…
Natural Language InferenceEvaluating Robustness of Generative Search Engine on Adversarial Factual Questions
Generative search engines have the potential to transform how people seek information online, but generated responses from existing large language models (LLMs)-backed generative search engines may not always be accurate…
RetrievalRetrieval-augmented Generation