Domain Fine-Tuning vs. Retrieval-Augmented Generation for Medical Multiple-Choice Question Answering: A Controlled Comparison at the 4B-Parameter Scale

2026-04-26 · Avi-ad Avraam Buskila arxiv

Practitioners deploying small open-weight large language models (LLMs) for medical question answering face a recurring design choice: invest in a domain-fine-tuned model, or keep a general-purpose model and inject domain knowledge at inference time via retrieval-augmented generation (RAG). We isolate this trade-off by holding model size, prompt template, decoding temperature, retrieval pipeline, and evaluation protocol fixed, and varying only (i) whether the model has been domain-adapted (Gemma 3 4B vs. MedGemma 4B, both 4-bit quantized and served via Ollama) and (ii) whether retrieved passages from a medical knowledge corpus are inserted into the prompt. We evaluate all four cells of this 2x2 design on the full MedQA-USMLE 4-option test split (1,273 questions) with three repetitions per question (15,276 LLM calls). Domain fine-tuning yields a +6.8 percentage-point gain in majority-vote accuracy over the general 4B baseline (53.3% vs. 46.4%, McNemar p < 10^-4). RAG over MedMCQA explanations does not produce a statistically significant gain in either model, and in the domain-tuned model the point estimate is slightly negative (-1.9 pp, p = 0.16). At this scale and on this benchmark, domain knowledge encoded in weights dominates domain knowledge supplied in context. We release the full experiment code and JSONL traces to support replication.

📄 PDF Abstract BibTeX arXiv:2604.23801

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering

ALoFTRAG: Automatic Local Fine Tuning for Retrieval Augmented Generation

2025-01-21 · Peter Devine

Retrieval Augmented Generation (RAG) systems have been shown to improve the accuracy of Large Language Model (LLM) outputs. However, these models can often achieve low accuracy when applied to new data domains. We introd…

Language Modeling Language Modelling Large Language Model RAG+2

ORPHEAS: A Cross-Lingual Greek-English Embedding Model for Retrieval-Augmented Generation

2026-04-22 · Ioannis E. Livieris, Athanasios Koursaris, Alexandra Apostolopoulou, Konstantinos Kanaris Dimitris Tsakalidis 외 arxiv

Effective retrieval-augmented generation across bilingual Greek--English applications requires embedding models capable of capturing both domain-specific semantic relationships and cross-lingual semantic alignment. Exist…

Document Retrieval Augmented Fine-Tuning (DRAFT) for safety-critical software assessments

2025-05-02 · Regan Bolton, Mohammadreza Sheikhfathollahi, Simon Parkinson, Vanessa Vulovic 외

Safety critical software assessment requires robust assessment against complex regulatory frameworks, a process traditionally limited by manual evaluation. This paper presents Document Retrieval-Augmented Fine-Tuning (DR…

Dataset Generation Language Modeling Language Modelling Large Language Model+3

Model-agnostic Retrieval-Augmented Extended Forecasting for time series

2026-08-14 · Juan Pablo Villa Serna, Rohan Asthana, Vasileios Belagiannis arxiv

Time series forecasting with pretrained foundation models has demonstrated strong zero-shot capabilities. However, achieving optimal performance on time series with short or negligible historical data in domain-specific …

Time Series Forecasting Domain Adaptation

MMRAG-RFT: Two-stage Reinforcement Fine-tuning for Explainable Multi-modal Retrieval-augmented Generation

2025-12-19 · Shengwei Zhao, Jingwen Yao, Sitong Wei, Linhai Xu 외 arxiv

Multi-modal Retrieval-Augmented Generation (MMRAG) enables highly credible generation by integrating external multi-modal knowledge, thus demonstrating impressive performance in complex multi-modal scenarios. However, ex…

Reinforcement Learning Response Generation Answer Generation