Combating False Negatives in Adversarial Imitation Learning
In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the desired behavior. However, as the trained policy learns to be more successful, the negative examples (the ones produced by the agent) become increasingly similar to expert ones. Despite the fact that the task is successfully accomplished in some of the agent's trajectories, the discriminator is trained to output low values for them. We hypothesize that this inconsistent training signal for the discriminator can impede its learning, and consequently leads to worse overall performance of the agent. We show experimental evidence for this hypothesis and that the 'False Negatives' (i.e. successful agent episodes) significantly hinder adversarial imitation learning, which is the first contribution of this paper. Then, we propose a method to alleviate the impact of false negatives and test it on the BabyAI environment. This method consistently improves sample efficiency over the baselines by at least an order of magnitude.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Integrating Language Guidance Into Image-Text Matching for Correcting False Negatives
Image-Text Matching (ITM) aims to establish the correspondence between images and sentences. ITM is fundamental to various vision and language understanding tasks. However, there are limitations in the way existing ITM b…
Cross-modal retrieval with noisy correspondenceImage-text matchingSentenceText MatchingCAPTURE: Context-Aware Prompt Injection Testing and Robustness Enhancement
Prompt injection remains a major security risk for large language models. However, the efficacy of existing guardrail models in context-aware settings remains underexplored, as they often rely on static attack benchmarks…
TuckerDNCaching: high-quality negative sampling with tucker decomposition
Knowledge Graph Embedding (KGE) translates entities and relations of knowledge graphs (KGs) into a low-dimensional vector space, enabling an efficient way of predicting missing facts. Generally, KGE models are trained wi…
Graph EmbeddingKnowledge Graph EmbeddingKnowledge GraphsLink PredictionA new wave of vehicle insurance fraud fueled by generative AI
Generative AI is supercharging insurance fraud by making it easier to falsify accident evidence at scale and in rapid time. Insurance fraud is a pervasive and costly problem, amounting to tens of billions of dollars in l…
DeepFake DetectionVideo GenerationMAFA: Managing False Negatives for Vision-Language Pre-training
We consider a critical issue of false negatives in Vision-Language Pre-training (VLP), a challenge that arises from the inherent many-to-many correspondence of image-text pairs in large-scale web-crawled datasets. The pr…