Exploring Uncertainty in Conditional Multi-Modal Retrieval Systems
We cast visual retrieval as a regression problem by posing triplet loss as a regression loss. This enables epistemic uncertainty estimation using dropout as a Bayesian approximation framework in retrieval. Accordingly, Monte Carlo (MC) sampling is leveraged to boost retrieval performance. Our approach is evaluated on two applications: person re-identification and autonomous car driving. Comparable state-of-the-art results are achieved on multiple datasets for the former application. We leverage the Honda driving dataset (HDD) for autonomous car driving application. It provides multiple modalities and similarity notions for ego-motion action understanding. Hence, we present a multi-modal conditional retrieval network. It disentangles embeddings into separate representations to encode different similarities. This form of joint learning eliminates the need to train multiple independent networks without any performance degradation. Quantitative evaluation highlights our approach competence, achieving 6% improvement in a highly uncertain environment.
Code (0)
등록된 구현이 없습니다.
Tasks
Action UnderstandingPerson Re-IdentificationregressionRetrievalTripletMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fuzzy Multimodal Learning for Trusted Cross-modal Retrieval
Cross-modal retrieval aims to match related samples across distinct modalities, facilitating the retrieval and discovery of heterogeneous information. Although existing methods show promising performance, most are de…
Cross-Modal RetrievalRetrievalUncertainty QuantificationGeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding
Standard dual-encoder vision-language models that map images and text to deterministic points on a shared unit hypersphere through $\ell_2$ normalization typically expose neither \emph{aleatoric} uncertainty (cross-modal…
To Retrieve or Not to Retrieve? Uncertainty Detection for Dynamic Retrieval Augmented Generation
Retrieval-Augmented Generation equips large language models with the capability to retrieve external knowledge, thereby mitigating hallucinations by incorporating information beyond the model's intrinsic abilities. Howev…
Long Form Question AnsweringQuestion AnsweringRetrievalRetrieval-augmented GenerationProbabilistic Multimodal Representation Learning
Learning multimodal representations is a requirement for many tasks such as image--caption retrieval. Previous work on this problem has only focused on finding good vector representations without any explicit measure of …
Representation LearningRetrievalUncertainty-Aware Alignment Network for Cross-Domain Video-Text Retrieval
Video-text retrieval is an important but challenging research task in the multimedia community. In this paper, we address the challenge task of Unsupervised Domain Adaptation Video-text Retrieval (UDAVR), assuming that …
Domain AdaptationRetrievalText RetrievalUnsupervised Domain Adaptation+1