Adversarial Contrastive Estimation
Learning by contrasting positive and negative samples is a general strategy adopted by many methods. Noise contrastive estimation (NCE) for word embeddings and translating embeddings for knowledge graphs are examples in NLP employing this approach. In this work, we view contrastive learning as an abstraction of all such methods and augment the negative sampler into a mixture distribution containing an adversarially learned sampler. The resulting adaptive sampler finds harder negative examples, which forces the main model to learn a better representation of the data. We evaluate our proposal on learning word embeddings, order embeddings and knowledge graph embeddings and observe both faster convergence and improved results on multiple metrics.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningKnowledge Graph EmbeddingsKnowledge GraphsLearning Word EmbeddingsWord EmbeddingsSimilar Papers 제목 키워드 기반
Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks
Monocular Depth Estimation (MDE) plays a vital role in applications such as autonomous driving. However, various attacks target MDE models, with physical attacks posing significant threats to system security. Traditional…
Adversarial RobustnessAutonomous DrivingContrastive LearningDepth Estimation+1Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks
Monocular Depth Estimation (MDE) is a critical component in applications such as autonomous driving. There are various attacks against MDE networks. These attacks, especially the physical ones, pose a great threat to the…
Adversarial RobustnessAutonomous DrivingContrastive LearningDepth Estimation+1Exponential Family Estimation via Adversarial Dynamics Embedding
We present an efficient algorithm for maximum likelihood estimation (MLE) of exponential family models, with a general parametrization of the energy function that includes neural networks. We exploit the primal-dual view…
Contrastive Weighted Learning for Near-Infrared Gaze Estimation
Appearance-based gaze estimation has been very successful with the use of deep learning. Many following works improved domain generalization for gaze estimation. However, even though there has been much progress in domai…
Adversarial AttackContrastive LearningData AugmentationDomain Generalization+2Dihedral angle prediction using generative adversarial networks
Several dihedral angles prediction methods were developed for protein structure prediction and their other applications. However, distribution of predicted angles would not be similar to that of real angles. To address t…
Density EstimationGenerative Adversarial NetworkPredictionProtein Structure Prediction+1