A Framework for Evaluating the Retrieval Effectiveness of Search Engines
This chapter presents a theoretical framework for evaluating next generation search engines. We focus on search engines whose results presentation is enriched with additional information and does not merely present the usual list of 10 blue links, that is, of ten links to results, accompanied by a short description. While Web search is used as an example here, the framework can easily be applied to search engines in any other area. The framework not only addresses the results presentation, but also takes into account an extension of the general design of retrieval effectiveness tests. The chapter examines the ways in which this design might influence the results of such studies and how a reliable test is best designed.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalSimilar Papers 제목 키워드 기반
Evaluating Robustness of Generative Search Engine on Adversarial Factual Questions
Generative search engines have the potential to transform how people seek information online, but generated responses from existing large language models (LLMs)-backed generative search engines may not always be accurate…
RetrievalRetrieval-augmented GenerationEvaluating Search Engines and Large Language Models for Answering Health Questions
Search engines (SEs) have traditionally been primary tools for information seeking, but the new Large Language Models (LLMs) are emerging as powerful alternatives, particularly for question-answering tasks. This study co…
MisinformationNavigateQuestion AnsweringRAG+1Evaluating Fairness in Argument Retrieval
Existing commercial search engines often struggle to represent different perspectives of a search query. Argument retrieval systems address this limitation of search engines and provide both positive (PRO) and negative (…
Argument RetrievalDiversityFairnessRetrievalThe retrieval effectiveness of search engines on navigational queries
Purpose - To test major Web search engines on their performance on navigational queries, i.e. searches for homepages. Design/methodology/approach - 100 real user queries are posed to six search engines (Google, Yahoo, MS…
RetrievalvalidThe Retrieval Effectiveness of Web Search Engines: Considering Results Descriptions
Purpose: To compare five major Web search engines (Google, Yahoo, MSN, Ask.com, and Seekport) for their retrieval effectiveness, taking into account not only the results but also the results descriptions. Design/Method…
Retrieval