Assaying on the Robustness of Zero-Shot Machine-Generated Text Detectors
To combat the potential misuse of Natural Language Generation (NLG) technology, a variety of algorithms have been developed for the detection of AI-generated texts. Traditionally, this task is treated as a binary classification problem. Although supervised learning has demonstrated promising results, acquiring labeled data for detection purposes poses real-world challenges and the risk of overfitting. In an effort to address these issues, we delve into the realm of zero-shot machine-generated text detection. Existing zero-shot detectors, typically designed for specific tasks or topics, often assume uniform testing scenarios, limiting their practicality. In our research, we explore various advanced Large Language Models (LLMs) and their specialized variants, contributing to this field in several ways. In empirical studies, we uncover a significant correlation between topics and detection performance. Secondly, we delve into the influence of topic shifts on zero-shot detectors. These investigations shed light on the adaptability and robustness of these detection methods across diverse topics. The code is available at \url{https://github.com/yfzhang114/robustness-detection}.
Code (1)
Tasks
Binary ClassificationText DetectionText GenerationSimilar Papers 제목 키워드 기반
A Machine Learning Approach for Material Type Logging and Chemical Assaying from Autonomous Measure-While-Drilling (MWD) Data
Understanding the structure and mineralogical composition of a region is an essential step in mining, both during exploration (before mining) and in the mining process. During exploration, sparse but high-quality data ar…
Zero-Shot Detection of Machine-Generated Codes
This work proposes a training-free approach for the detection of LLMs-generated codes, mitigating the risks associated with their indiscriminate usage. To the best of our knowledge, our research is the first to investiga…
Language ModellingText DetectionFindings of the WMT 2020 Shared Task on Machine Translation Robustness
We report the findings of the second edition of the shared task on improving robustness in Machine Translation (MT). The task aims to test current machine translation systems in their ability to handle challenges facing …
DiversityMachine TranslationTranslationWho Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore
The efficacy of an large language model (LLM) generated text detector depends substantially on the availability of sizable training data. White-box zero-shot detectors, which require no such data, are nonetheless limited…
Language ModelingLanguage ModellingLarge Language ModelLLM-generated Text Detection+1Once a Response, Always a Response: Detecting LLM-generated Text via Latent Prompt Restoration
Large language models (LLMs) can generate fluent and convincing text at scale, creating growing risks for misinformation dissemination, educational misuse, and platform governance. These concerns make robust detection of…