Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models
The generation of undesirable and factually incorrect content of large language models poses a significant challenge and remains largely an unsolved issue. This paper studies the integration of a contrastive learning objective for fine-tuning LLMs for implicit knowledge editing and controlled text generation. Optimizing the training objective entails aligning text perplexities in a contrastive fashion. To facilitate training the model in a self-supervised fashion, we leverage an off-the-shelf LLM for training data generation. We showcase applicability in the domain of detoxification. Herein, the proposed approach leads to a significant decrease in the generation of toxic content while preserving general utility for downstream tasks such as commonsense reasoning and reading comprehension. The proposed approach is conceptually simple but empirically powerful.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive Learningknowledge editingReading ComprehensionText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CFL: Causally Fair Language Models Through Token-level Attribute Controlled Generation
We propose a method to control the attributes of Language Models (LMs) for the text generation task using Causal Average Treatment Effect (ATE) scores and counterfactual augmentation. We explore this method, in the conte…
AttributecounterfactualText GenerationExploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
Pre-trained language models (LMs) are shown to easily generate toxic language. In this work, we systematically explore domain-adaptive training to reduce the toxicity of language models. We conduct this study on three di…
Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimization
Warning: this paper contains model outputs exhibiting offensiveness and biases. Recently pre-trained language models (PLMs) have prospered in various natural language generation (NLG) tasks due to their ability to genera…
Text GenerationUniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
We present UniDetox, a universally applicable method designed to mitigate toxicity across various large language models (LLMs). Previous detoxification methods are typically model-specific, addressing only individual mod…
Dataset DistillationLanguage ModelingLanguage ModellingMulti-Attribute Controlled Text Generation with Contrastive-Generator and External-Discriminator
Though existing researches have achieved impressive results in controlled text generation, they focus mainly on single-attribute control. However, in applications like automatic comments, the topic and sentiment need to …
AttributeComment GenerationText Generation