Language-Enhanced Latent Representations for Out-of-Distribution Detection in Autonomous Driving
Out-of-distribution (OOD) detection is essential in autonomous driving, to determine when learning-based components encounter unexpected inputs. Traditional detectors typically use encoder models with fixed settings, thus lacking effective human interaction capabilities. With the rise of large foundation models, multimodal inputs offer the possibility of taking human language as a latent representation, thus enabling language-defined OOD detection. In this paper, we use the cosine similarity of image and text representations encoded by the multimodal model CLIP as a new representation to improve the transparency and controllability of latent encodings used for visual anomaly detection. We compare our approach with existing pre-trained encoders that can only produce latent representations that are meaningless from the user's standpoint. Our experiments on realistic driving data show that the language-based latent representation performs better than the traditional representation of the vision encoder and helps improve the detection performance when combined with standard representations.
Code (0)
등록된 구현이 없습니다.
Tasks
Anomaly DetectionAutonomous DrivingOut-of-Distribution DetectionOut of Distribution (OOD) DetectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DiffLM: Controllable Synthetic Data Generation via Diffusion Language Models
Recent advancements in large language models (LLMs) have significantly enhanced their knowledge and generative capabilities, leading to a surge of interest in leveraging LLMs for high-quality data synthesis. However, syn…
Prompt EngineeringSynthetic Data GenerationTraining Heterogeneous Features in Sequence to Sequence Tasks: Latent Enhanced Multi-filter Seq2Seq Model
In language processing, training data with extremely large variance may lead to difficulty in the language model's convergence. It is difficult for the network parameters to adapt sentences with largely varied semantics …
ClusteringDecoderMachine TranslationQuestion Answering+3Detect Camouflaged Spam Content via StoneSkipping: Graph and Text Joint Embedding for Chinese Character Variation Representation
The task of Chinese text spam detection is very challenging due to both glyph and phonetic variations of Chinese characters. This paper proposes a novel framework to jointly model Chinese variational, semantic, and conte…
Chinese Spam DetectionGraph EmbeddingLanguage ModelingLanguage Modelling+1Scaling Pretrained Representations Enables Label-Free Out-of-Distribution Detection Without Fine-Tuning
Models trained with deep learning often fail to signal when inputs fall outside their training data manifold, leading to unreliable predictions under distribution shift. Prior work suggests that effective out-of-distribu…
Out-of-Distribution DetectionToward a Deep Neural Approach for Knowledge-Based IR
This paper tackles the problem of the semantic gap between a document and a query within an ad-hoc information retrieval task. In this context, knowledge bases (KBs) have already been acknowledged as valuable means since…
Ad-Hoc Information RetrievalDocument RankingImplicit RelationsInformation Retrieval+1