Comparative Analysis of Open-Source Language Models in Summarizing Medical Text Data

2024-05-25 · Yuhao Chen, Zhimu Wang, Bo Wen, Farhana Zulkernine

Unstructured text in medical notes and dialogues contains rich information. Recent advancements in Large Language Models (LLMs) have demonstrated superior performance in question answering and summarization tasks on unstructured text data, outperforming traditional text analysis approaches. However, there is a lack of scientific studies in the literature that methodically evaluate and report on the performance of different LLMs, specifically for domain-specific data such as medical chart notes. We propose an evaluation approach to analyze the performance of open-source LLMs such as Llama2 and Mistral for medical summarization tasks, using GPT-4 as an assessor. Our innovative approach to quantitative evaluation of LLMs can enable quality control, support the selection of effective LLMs for specific tasks, and advance knowledge discovery in digital health.

📄 PDF Abstract BibTeX arXiv:2405.16295

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Adam 설명 없음

Residual Connection 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

Multi-Head Attention 설명 없음

Automating Customer Needs Analysis: A Comparative Study of Large Language Models in the Travel Industry

2024-04-27 · Simone Barandoni, Filippo Chiarello, Lorenzo Cascone, Emiliano Marrale 외

In the rapidly evolving landscape of Natural Language Processing (NLP), Large Language Models (LLMs) have emerged as powerful tools for many tasks, such as extracting valuable insights from vast amounts of textual data. …

Multi-Agent Systems: From Classical Paradigms to Large Foundation Model-Enabled Futures

2026-04-20 · Zixiang Wang, Mengjia Gong, Qiyu Sun, Jing Xu 외 arxiv

With the rapid advancement of artificial intelligence, multi-agent systems (MASs) are evolving from classical paradigms toward architectures built upon large foundation models (LFMs). This survey provides a systematic re…

Comparing Approaches to Automatic Summarization in Less-Resourced Languages

2025-12-30 · Chester Palen-Michel, Constantine Lignos arxiv

Automatic text summarization has achieved high performance in high-resourced languages like English, but comparatively less attention has been given to summarization in less-resourced languages. This work compares a vari…

Text Summarization Data Augmentation

Zero-shot OCR Accuracy of Low-Resourced Languages: A Comparative Analysis on Sinhala and Tamil

2025-07-24 · Nevidu Jayatilleke, Nisansa de Silva arxiv

Solving the problem of Optical Character Recognition (OCR) on printed text for Latin and its derivative scripts can now be considered settled due to the volumes of research done on English and other High-Resourced Langua…

Document AI

Closing the gap between open-source and commercial large language models for medical evidence summarization

2024-07-25 · Gongbo Zhang, Qiao Jin, Yiliang Zhou, Song Wang 외

Large language models (LLMs) hold great promise in summarizing medical evidence. Most recent studies focus on the application of proprietary LLMs. Using proprietary LLMs introduces multiple risk factors, including a lack…

Model Selection