GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Dense retrieval approaches can overcome the lexical gap and lead to significantly improved search results. However, they require large amounts of training data which is not available for most domains. As shown in previous work (Thakur et al., 2021b), the performance of dense retrievers severely degrades under a domain shift. This limits the usage of dense retrieval approaches to only a few domains with large training datasets.In this paper, we propose the novel unsupervised domain adaptation method Generative Pseudo Labeling (GPL), which combines a query generator with pseudo labeling from a cross-encoder. On six representative domain-specialized datasets, we find the proposed GPL can outperform an out-of-the-box state-of-the-art dense retrieval approach by up to 8.9 points nDCG@10. GPL requires less (unlabeled) data from the target domain and is more robust in its training than previous methods.We further investigate the role of six recent pre-training methods in the scenario of domain adaptation for retrieval tasks, where only three could yield improved results. The best approach, TSDAE (Wang et al., 2021) can be combined with GPL, yielding another average improvement of 1.0 points nDCG@10 across the six tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationRetrievalUnsupervised Domain AdaptationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Dense retrieval approaches can overcome the lexical gap and lead to significantly improved search results. However, they require large amounts of training data which is not available for most domains. As shown in previou…
Domain AdaptationRetrievalUnsupervised Domain AdaptationZero-shot Text SearchATPL: Mutually enhanced adversarial training and pseudo labeling for unsupervised domain adaptation
Unsupervised domain adaptation aims to transfer knowledge from a labeled source domain to a related but unlabeled target domain. Most existing approaches either adversarially reduce the domain shift or use pseudo-labels …
Domain AdaptationUnsupervised Domain AdaptationSource-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling
Domain adaptation typically requires to access source domain data to utilize their distribution information for domain alignment with the target data. However, in many real-world scenarios, the source data may not be acc…
DenoisingDomain AdaptationImage SegmentationSemantic Segmentation+1Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling
Unsupervised domain adaptation aims to address the problem of classifying unlabeled samples from the target domain whilst labeled samples are only available from the source domain and the data distributions are different…
ClusteringDomain AdaptationStructured PredictionUnsupervised Domain AdaptationSelective Pseudo-Labeling with Reinforcement Learning for Semi-Supervised Domain Adaptation
Recent domain adaptation methods have demonstrated impressive improvement on unsupervised domain adaptation problems. However, in the semi-supervised domain adaptation (SSDA) setting where the target domain has a few lab…
Domain AdaptationQ-Learningreinforcement-learningReinforcement Learning (RL)+2