Building a Silver-Standard Dataset from NICE Guidelines for Clinical LLMs
Large language models (LLMs) are increasingly used in healthcare, yet standardised benchmarks for evaluating guideline-based clinical reasoning are missing. This study introduces a validated dataset derived from publicly available guidelines across multiple diagnoses. The dataset was created with the help of GPT and contains realistic patient scenarios, as well as clinical questions. We benchmark a range of recent popular LLMs to showcase the validity of our dataset. The framework supports systematic evaluation of LLMs' clinical utility and guideline adherence.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
UD-English-CHILDES: A Collected Resource of Gold and Silver Universal Dependencies Trees for Child Language Interactions
CHILDES is a widely used resource of transcribed child and child-directed speech. This paper introduces UD-English-CHILDES, the first officially released Universal Dependencies (UD) treebank derived from previously depen…
GRAIN-S: Manually Annotated Syntax for German Interviews
We present GRAIN-S, a set of manually created syntactic annotations for radio interviews in German. The dataset extends an existing corpus GRAIN and comes with constituency and dependency trees for six interviews. The ra…
Learning with Silver Standard Data for Zero-shot Relation Extraction
The superior performance of supervised relation extraction (RE) methods heavily relies on a large amount of gold standard data. Recent zero-shot relation extraction methods converted the RE task to other NLP tasks and us…
RelationRelation ExtractionBuilding a Synthetic Biomedical Research Article Citation Linkage Corpus
Citations are frequently used in publications to support the presented results and to demonstrate the previous discoveries while also assisting the reader in following the chronological progression of information through…
Semantic SimilaritySemantic Textual SimilaritySentenceSentence Embedding+1Pixel-level Counterfactual Contrastive Learning for Medical Image Segmentation
Image segmentation relies on large annotated datasets, which are expensive and slow to produce. Silver-standard (AI-generated) labels are easier to obtain, but they risk introducing bias. Self-supervised learning, needin…
Medical Image SegmentationSelf-Supervised LearningRepresentation LearningContrastive Learning