paper-with-me

Papers

Dynamic Gradient Sparsification Training for Few-Shot Fine-tuning of CT Lymph Node Segmentation Foundation Model

2025-03-02 · Zihao Luo, Zijun Gao, Wenjun Liao, Shichuan Zhang, Guotai Wang, Xiangde Luo

Accurate lymph node (LN) segmentation is critical in radiotherapy treatment and prognosis analysis, but is limited by the need for large annotated datasets. While deep learning-based segmentation foundation models show potential in developing high-performing models with fewer samples, their medical adaptation faces LN domain-specific prior deficiencies and inefficient few-shot fine-tuning for complex clinical practices, highlighting the necessity of an LN segmentation foundation model. In this work, we annotated 36,106 visible LNs from 3,346 publicly available head-and-neck CT scans to establish a robust LN segmentation model (nnUNetv2). Building on this, we propose Dynamic Gradient Sparsification Training (DGST), a few-shot fine-tuning approach that preserves foundational knowledge while dynamically updating the most critical parameters of the LN segmentation model with few annotations. We validate it on two publicly available LN segmentation datasets: SegRap2023 and LNQ2023. The results show that DGST outperforms existing few-shot fine-tuning methods, achieving satisfactory performance with limited labeled data. We release the dataset, models and all implementations to facilitate relevant research: https://github.com/Zihaoluoh/LN-Seg-FM.

📄 PDF Abstract BibTeX arXiv:2503.00748

Code (1)

zihaoluoh/ln-seg-fm 공식 구현 pytorch

Tasks

PrognosisSegmentation

Methods 이 논문이 사용한 방법론

Gradient Sparsification Gradient Sparsification is a technique for distributed training that sparsifies stochastic gradients to reduce the communication cost, with minor increase in the number of…

Similar Papers 제목 키워드 기반

Gradient Sparsification For \emph{Masked Fine-Tuning} of Transformers

2021-11-16 · ACL ARR November 2021 11 · Anonymous

Fine-tuning masked language models is widely adopted for transfer learning to downstream tasks and can be achieved by (1) freezing gradients of the pretrained network or only updating gradients of a newly added classific…

Transfer LearningXLM-R

ONG: One-Shot NMF-based Gradient Masking for Efficient Model Sparsification

2025-08-18 · Sankar Behera, Yamuna Prasad arxiv

Deep Neural Networks (DNNs) have achieved remarkable success but their large size poses deployment challenges. While various pruning techniques exist, many involve complex iterative processes, specialized criteria, or st…

Preserving Near-Optimal Gradient Sparsification Cost for Scalable Distributed Deep Learning

2024-02-21 · Daegun Yoon, Sangyoon Oh

Communication overhead is a major obstacle to scaling distributed training systems. Gradient sparsification is a potential optimization approach to reduce the communication volume without significant loss of model fideli…

TEDDY: Trimming Edges with Degree-based Discrimination strategY

2024-02-02 · Hyunjin Seo, Jihun Yun, Eunho Yang

Since the pioneering work on the lottery ticket hypothesis for graph neural networks (GNNs) was proposed in Chen et al. (2021), the study on finding graph lottery tickets (GLT) has become one of the pivotal focus in the …

CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification

2022-12-06 · Lirui Xiao, Huanrui Yang, Zhen Dong, Kurt Keutzer 외

Mixed-precision quantization has been widely applied on deep neural networks (DNNs) as it leads to significantly better efficiency-accuracy tradeoffs compared to uniform quantization. Meanwhile, determining the exact pre…

Quantization