Scissorhands: Scrub Data Influence via Connection Sensitivity in Networks
Machine unlearning has become a pivotal task to erase the influence of data from a trained model. It adheres to recent data regulation standards and enhances the privacy and security of machine learning applications. In this work, we present a new machine unlearning approach Scissorhands. Initially, Scissorhands identifies the most pertinent parameters in the given model relative to the forgetting data via connection sensitivity. By reinitializing the most influential top-k percent of these parameters, a trimmed model for erasing the influence of the forgetting data is obtained. Subsequently, Scissorhands fine-tunes the trimmed model with a gradient projection-based approach, seeking parameters that preserve information on the remaining data while discarding information related to the forgetting data. Our experimental results, conducted across image classification and image generation tasks, demonstrate that Scissorhands, showcases competitive performance when compared to existing methods. Source code is available at https://github.com/JingWu321/Scissorhands.
Code (1)
Tasks
image-classificationImage ClassificationImage GenerationMachine UnlearningSensitivitySimilar Papers 제목 키워드 기반
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Large language models(LLMs) have sparked a new wave of exciting AI applications. Hosting these models at scale requires significant memory resources. One crucial memory bottleneck for the deployment stems from the contex…
QuantizationFault-Tolerant Control Design in Scrubber Plant with Fault on Sensor Sensitivity
The concept of fault-tolerant control has extensively been explored with various mapping of development. It starts from the system characteristic, the robustness of the controller, estimation methods and optimization, to…
SensitivityEnterprise Disk Drive Scrubbing Based on Mondrian Conformal Predictors
Disk scrubbing is a process aimed at resolving read errors on disks by reading data from the disk. However, scrubbing the entire storage array at once can adversely impact system performance, particularly during periods …
Conformal PredictionAnalyzing Leakage of Personally Identifiable Information in Language Models
Language Models (LMs) have been shown to leak information about training data through sentence-level membership inference and reconstruction attacks. Understanding the risk of LMs leaking Personally Identifiable Informat…
SentenceContractScrub: A benchmark for final review of legal contracts
Legal work, with its heavy reliance on processing large amounts of text, is often considered one of the domains most exposed to the use of LLMs. Contract ``scrubbing,'' the final review of transactional agreements for er…