Building Evaluation Datasets for Consumer-Oriented Information Retrieval
Common people often experience difficulties in accessing relevant, correct, accurate and understandable health information online. Developing search techniques that aid these information needs is challenging. In this paper we present the datasets created by CLEF eHealth Lab from 2013-2015 for evaluation of search solutions to support common people finding health information online. Specifically, the CLEF eHealth information retrieval (IR) task of this Lab has provided the research community with benchmarks for evaluating consumer-centered health information retrieval, thus fostering research and development aimed to address this challenging problem. Given consumer queries, the goal of the task is to retrieve relevant documents from the provided collection of web pages. The shared datasets provide a large health web crawl, queries representing people{'}s real world information needs, and relevance assessment judgements for the queries.
Code (0)
등록된 구현이 없습니다.
Tasks
Information RetrievalRetrievalSimilar Papers 제목 키워드 기반
PerMedCQA: Benchmarking Large Language Models on Medical Consumer Question Answering in Persian Language
Medical consumer question answering (CQA) is crucial for empowering patients by providing personalized and reliable health information. Despite recent advances in large language models (LLMs) for medical QA, consumer-ori…
BenchmarkingQuestion AnsweringWe Need to Rethink How We Describe and Organize Spatial Information: Instrumenting and Observing the Community of Users to Improve Data Description and Discovery
In Spatial Data Infrastructure or Cyber Infrastructure, the description of geographic data semantics is intended to support data discovery, reuse and integration. In the vast majority of cases the producers of these data…
Redefining Retrieval Evaluation in the Era of LLMs
Traditional Information Retrieval (IR) metrics, such as nDCG, MAP, and MRR, assume that human users sequentially examine documents with diminishing attention to lower ranks. This assumption breaks down in Retrieval Augme…
Information RetrievalOther Roles Matter! Enhancing Role-Oriented Dialogue Summarization via Role Interactions
Role-oriented dialogue summarization is to generate summaries for different roles in the dialogue, e.g., merchants and consumers. Existing methods handle this task by summarizing each role's content separately and thus a…
DecoderEnhancing Polygonal Building Segmentation via Oriented Corners
The growing demand for high-resolution maps across various applications has underscored the necessity of accurately segmenting building vectors from overhead imagery. However, current deep neural networks often produce r…