Make Text Unlearnable: Exploiting Effective Patterns to Protect Personal Data
This paper addresses the ethical concerns arising from the use of unauthorized public data in deep learning models and proposes a novel solution. Specifically, building on the work of Huang et al. (2021), we extend their bi-level optimization approach to generate unlearnable text using a gradient-based search technique. However, although effective, this approach faces practical limitations, including the requirement of batches of instances and model architecture knowledge that is not readily accessible to ordinary users with limited access to their own data. Furthermore, even with semantic-preserving constraints, unlearnable noise can alter the text's semantics. To address these challenges, we extract simple patterns from unlearnable text produced by bi-level optimization and demonstrate that the data remains unlearnable for unknown models. Additionally, these patterns are not instance- or dataset-specific, allowing users to readily apply them to text classification and question-answering tasks, even if only a small proportion of users implement them on their public content. We also open-source codes to generate unlearnable text and assess unlearnable noise to benefit the public and future studies.
Code (1)
Tasks
Question Answeringtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Unlearnable Text for Neural Classifiers
Neural text classification models are known to001explore statistical patterns during supervised002learning. However, such patterns include spurious patterns and superficial regularity in the004training data. In this …
ClassificationGender Classificationtext-classificationText Classification+1Flew Over Learning Trap: Learn Unlearnable Samples by Progressive Staged Training
Unlearning techniques are proposed to prevent third parties from exploiting unauthorized data, which generate unlearnable samples by adding imperceptible perturbations to data for public publishing. These unlearnable sam…
Perturbation-Induced Linearization: Constructing Unlearnable Data with Solely Linear Classifiers
Collecting web data to train deep models has become increasingly common, raising concerns about unauthorized data usage. To mitigate this issue, unlearnable examples introduce imperceptible perturbations into data, preve…
UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation
Image segmentation is a crucial vision task that groups pixels within an image into semantically meaningful segments, which is pivotal in obtaining a fine-grained understanding of real-world scenes. However, an increasin…
AllBilevel OptimizationImage SegmentationInteractive Segmentation+2Transferable Unlearnable Examples
With more people publishing their personal data online, unauthorized data usage has become a serious concern. The unlearnable strategies have been introduced to prevent third parties from training on the data without per…