From Data Leak to Secret Misses: The Impact of Data Leakage on Secret Detection Models

2026-01-30 · Farnaz Soltaniani, Mohammad Ghafari arxiv

Machine learning models are increasingly used for software security tasks. These models are commonly trained and evaluated on large Internet-derived datasets, which often contain duplicated or highly similar samples. When such samples are split across training and test sets, data leakage may occur, allowing models to memorize patterns instead of learning to generalize. We investigate duplication in a widely used benchmark dataset of hard coded secrets and show how data leakage can substantially inflate the reported performance of AI-based secret detectors, resulting in a misleading picture of their real-world effectiveness.

📄 PDF Abstract BibTeX arXiv:2601.22946

Code (0)

등록된 구현이 없습니다.

Can You Keep a Secret? Involuntary Information Leakage in Language Model Writing

2026-05-11 · Ari Holtzman, Peter West arxiv

Language models are deployed in settings that require compartmentalization: system prompts should not be disclosed, chain-of-thought reasoning is hidden from users, and sensitive data passes through shared contexts. We t…

Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective

2026-04-20 · Meifang Chen, Zhe Yang, Huang Nianchen, Yizhan Huang 외 arxiv

Code secrets are sensitive assets for software developers, and their leakage poses significant cybersecurity risks. While the rapid development of AI code assistants powered by Code Large Language Models (CLLMs), CLLMs a…

Inadvertent Context Leakage in Language Models

2026-08-20 · Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri, Sanjam Garg 외 arxiv

For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data. We study whether the mere presence of such secrets in a model's con…

Robust Skin Color Driven Privacy Preserving Face Recognition via Function Secret Sharing

2024-07-06 · Dong Han, Yufan Jiang, Yong Li, Ricardo Mendes 외

In this work, we leverage the pure skin color patch from the face image as the additional information to train an auxiliary skin color feature extractor and face recognition model in parallel to improve performance of st…

Face Recognition Generative Adversarial Network Image Restoration Privacy Preserving

SecretGen: Privacy Recovery on Pre-Trained Models via Distribution Discrimination

2022-07-25 · Zhuowen Yuan, Fan Wu, Yunhui Long, Chaowei Xiao 외

Transfer learning through the use of pre-trained models has become a growing trend for the machine learning community. Consequently, numerous pre-trained models are released online to facilitate further research. However…

Model Selection Transfer Learning