Loss-Guided Auxiliary Agents for Overcoming Mode Collapse in GFlowNets
Although Generative Flow Networks (GFlowNets) are designed to capture multiple modes of a reward function, they often suffer from mode collapse in practice, getting trapped in early discovered modes and requiring prolonged training to find diverse solutions. Existing exploration techniques may rely on heuristic novelty signals. We propose Loss-Guided GFlowNets (LGGFN), a novel approach where an auxiliary GFlowNet's exploration is directly driven by the main GFlowNet's training loss. By prioritizing trajectories where the main model exhibits high loss, LGGFN focuses sampling on poorly understood regions of the state space. This targeted exploration significantly accelerates the discovery of diverse, high-reward samples. Empirically, across various benchmarks including grid environments, structured sequence generation, and Bayesian structure learning, LGGFN consistently enhances exploration efficiency and sample diversity compared to baselines. For instance, on a challenging sequence generation task, it discovered over 40 times more unique valid modes while simultaneously reducing the exploration error metric by approximately 99\%.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityvalidSimilar Papers 제목 키워드 기반
A Self-Supervised Auxiliary Loss for Deep RL in Partially Observable Settings
In this work we explore an auxiliary loss useful for reinforcement learning in environments where strong performing agents are required to be able to navigate a spatial environment. The auxiliary loss proposed is to mini…
NavigateSpatial ReasoningSemi-supervised Crowd Counting via Density Agency
In this paper, we propose a new agency-guided semi-supervised counting approach. First, we build a learnable auxiliary structure, namely the density agency to bring the recognized foreground regional features close to co…
Contrastive LearningCrowd CountingOvercoming Referential Ambiguity in Language-Guided Goal-Conditioned Reinforcement Learning
Teaching an agent to perform new tasks using natural language can easily be hindered by ambiguities in interpretation. When a teacher provides an instruction to a learner about an object by referring to its features, the…
Objectreinforcement-learningReinforcement LearningReinforcement Learning (RL)GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks
While leveraging additional training data is well established to improve adversarial robustness, it incurs the unavoidable cost of data collection and the heavy computation to train models. To mitigate the costs, we prop…
Adversarial RobustnessData AugmentationMulti-Task LearningCAREL: Instruction-guided reinforcement learning with cross-modal auxiliary objectives
Grounding the instruction in the environment is a key step in solving language-guided goal-reaching reinforcement learning problems. In automated reinforcement learning, a key concern is to enhance the model's ability to…
reinforcement-learningReinforcement LearningRetrievalSystematic Generalization+2