FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework

2025-02-27 · S M Sarwar

With the increasing prevalence of mental health conditions worldwide, AI-powered chatbots and conversational agents have emerged as accessible tools to support mental health. However, deploying Large Language Models (LLMs) in mental healthcare applications raises significant privacy concerns, especially regarding regulations like HIPAA and GDPR. In this work, we propose FedMentalCare, a privacy-preserving framework that leverages Federated Learning (FL) combined with Low-Rank Adaptation (LoRA) to fine-tune LLMs for mental health analysis. We investigate the performance impact of varying client data volumes and model architectures (e.g., MobileBERT and MiniLM) in FL environments. Our framework demonstrates a scalable, privacy-aware approach for deploying LLMs in real-world mental healthcare scenarios, addressing data security and computational efficiency challenges.

📄 PDF Abstract BibTeX arXiv:2503.05786

Code (0)

등록된 구현이 없습니다.

Tasks

Computational Efficiency Federated LearningPrivacy Preserving

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Multi-Head Attention 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

MobileBERT MobileBERT is a type of inverted-bottleneck BERT that compresses and accelerates the popular BERT model. MobileBERT is a thin version…

GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models

2025-07-06 · Kai Yao, Zhaorui Tan, Penglei Gao, Lichun Li 외 arxiv

The rapid growth of large language models (LLMs) with traditional centralized fine-tuning emerges as a key technique for adapting these models to domain-specific challenges, yielding privacy risks for both model and data…

Unintended Memorization of Sensitive Information in Fine-Tuned Language Models

2026-01-24 · Marton Szep, Jorge Marin Ruiz, Georgios Kaissis, Paulina Seidl 외 arxiv

Fine-tuning Large Language Models (LLMs) on sensitive datasets carries a substantial risk of unintended memorization and leakage of Personally Identifiable Information (PII), which can violate privacy regulations and com…

Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs

2025-03-16 · Bowen Tan, Zheng Xu, Eric Xing, Zhiting Hu 외

Synthetic data offers a promising path to train models while preserving data privacy. Differentially private (DP) finetuning of large language models (LLMs) as data generator is effective, but is impractical when computa…

Clustering Privacy Preserving Prompt Engineering

Provably Protecting Fine-Tuned LLMs from Training Data Extraction while Preserving Utility

2026-01-31 · Tom Segal, Asaf Shabtai, Yuval Elovici arxiv

Fine-tuning large language models (LLMs) on sensitive datasets raises privacy concerns, as training data extraction (TDE) attacks can expose highly confidential information. Existing defenses against such attacks either …

CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs

2026-05-15 · Kamil Guttmann, Zofia Fraś, Artur Nowakowski, Krzysztof Jassem arxiv

Current state-of-the-art Quality Estimation (QE) in machine translation relies on massive, proprietary LLMs, raising data privacy concerns. We demonstrate that smaller, open-source LLMs (<30B parameters) are a viable, co…

Machine Translation