Robust Learning Through Cross-Task Consistency
Visual perception entails solving a wide set of tasks (e.g., object detection, depth estimation, etc). The predictions made for different tasks out of one image are not independent, and therefore, are expected to be 'consistent'. We propose a flexible and fully computational framework for learning while enforcing Cross-Task Consistency (X-TAC). The proposed formulation is based on 'inference path invariance' over an arbitrary graph of prediction domains. We observe that learning with cross-task consistency leads to more accurate predictions, better generalization to out-of-distribution samples, and improved sample efficiency. This framework also leads to a powerful unsupervised quantity, called 'Consistency Energy, based on measuring the intrinsic consistency of the system. Consistency Energy well correlates with the supervised error (r=0.67), thus it can be employed as an unsupervised robustness metric as well as for detection of out-of-distribution inputs (AUC=0.99). The evaluations were performed on multiple datasets, including Taskonomy, Replica, CocoDoom, and ApolloScape.
Code (1)
Tasks
3D ReconstructionDepth EstimationMonocular Depth Estimationobject-detectionObject DetectionSingle-View 3D ReconstructionSurface Normals EstimationSimilar Papers 제목 키워드 기반
Robust Learning Through Cross-Task Consistency
Visual perception entails solving a wide set of tasks, e.g., object detection, depth estimation, etc. The predictions made for multiple tasks from the same image are not independent, and therefore, are expected to be con…
3D ReconstructionDepth EstimationMulti-Task Learningobject-detection+2TriangleNet: Edge Prior Augmented Network for Semantic Segmentation through Cross-Task Consistency
This paper addresses the task of semantic segmentation in computer vision, aiming to achieve precise pixel-wise classification. We investigate the joint training of models for semantic edge detection and semantic segment…
Autonomous DrivingEdge DetectionMulti-Task LearningReal-Time Semantic Segmentation+2Batched Self-Consistency Improves LLM Relevance Assessment and Ranking
Given some information need, Large Language Models (LLMs) are increasingly used for candidate text relevance assessment, typically using a one-by-one pointwise (PW) strategy where each LLM call evaluates one candidate at…
Passage RetrievalOn the Entity-Level Alignment in Crosslingual Consistency
Multilingual large language models (LLMs) are expected to recall factual knowledge consistently across languages. However, the factors that give rise to such crosslingual consistency -- and its frequent failure -- remain…
Entity AlignmentSocrates-Mol: Self-Oriented Cognitive Reasoning through Autonomous Trial-and-Error with Empirical-Bayesian Screening for Molecules
Molecular property prediction is fundamental to chemical engineering applications such as solvent screening. We present Socrates-Mol, a framework that transforms language models into empirical Bayesian reasoners through …
Molecular Property Prediction