Fluent dreaming for language models
Feature visualization, also known as "dreaming", offers insights into vision models by optimizing the inputs to maximize a neuron's activation or other internal component. However, dreaming has not been successfully applied to language models because the input space is discrete. We extend Greedy Coordinate Gradient, a method from the language model adversarial attack literature, to design the Evolutionary Prompt Optimization (EPO) algorithm. EPO optimizes the input prompt to simultaneously maximize the Pareto frontier between a chosen internal feature and prompt fluency, enabling fluent dreaming for language models. We demonstrate dreaming with neurons, output logits and arbitrary directions in activation space. We measure the fluency of the resulting prompts and compare language model dreaming with max-activating dataset examples. Critically, fluent dreaming allows automatically exploring the behavior of model internals in reaction to mildly out-of-distribution prompts. Code for running EPO is available at https://github.com/Confirm-Solutions/dreamy. A companion page demonstrating code usage is at https://confirmlabs.org/posts/dreamy.html
Code (1)
Tasks
Adversarial AttackLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Finding the DeepDream for Time Series: Activation Maximization for Univariate Time Series
Understanding how models process and interpret time series data remains a significant challenge in deep learning to enable applicability in safety-critical areas such as healthcare. In this paper, we introduce Sequence D…
Decision MakingTime SeriesTime Series ClassificationDreamingV2: Reinforcement Learning with Discrete World Models without Reconstruction
The present paper proposes a novel reinforcement learning method with world models, DreamingV2, a collaborative extension of DreamerV2 and Dreaming. DreamerV2 is a cutting-edge model-based reinforcement learning from pix…
Contrastive LearningModel-based Reinforcement Learningreinforcement-learningReinforcement Learning+1Multi-View Dreaming: Multi-View World Model with Contrastive Learning
In this paper, we propose Multi-View Dreaming, a novel reinforcement learning agent for integrated recognition and control from multi-view observations by extending Dreaming. Most current reinforcement learning method as…
Contrastive Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Learning Real-World Robot Policies by Dreaming
Learning to control robots directly based on images is a primary challenge in robotics. However, many existing reinforcement learning approaches require iteratively obtaining millions of robot samples to learn a policy, …
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Functional neuroimaging of psychedelic experience: An overview of psychological and neural effects and their relevance to research on creativity, daydreaming, and dreaming
Humans have employed an incredible variety of plant-derived substances over the millennia in order to alter consciousness and perception. Among the innumerable narcotics, analgesics, 'ordeal' drugs, and other psychoactiv…