Universal Instance Perception as Object Discovery and Retrieval
All instance perception tasks aim at finding certain objects specified by some queries such as category names, language expressions, and target annotations, but this complete field has been split into multiple independent subtasks. In this work, we present a universal instance perception model of the next generation, termed UNINEXT. UNINEXT reformulates diverse instance perception tasks into a unified object discovery and retrieval paradigm and can flexibly perceive different types of objects by simply changing the input prompts. This unified formulation brings the following benefits: (1) enormous data from different tasks and label vocabularies can be exploited for jointly training general instance-level representations, which is especially beneficial for tasks lacking in training data. (2) the unified model is parameter-efficient and can save redundant computation when handling multiple tasks simultaneously. UNINEXT shows superior performance on 20 challenging benchmarks from 10 instance-level tasks including classical image-level tasks (object detection and instance segmentation), vision-and-language tasks (referring expression comprehension and segmentation), and six video-level object tracking tasks. Code is available at https://github.com/MasterBin-IIAU/UNINEXT.
Code (1)
Tasks
Described Object DetectionGeneralized Referring Expression ComprehensionInstance SegmentationMulti-Object Tracking and SegmentationMultiple Object TrackingObjectobject-detectionObject DetectionObject DiscoveryObject TrackingReferring ExpressionReferring Expression ComprehensionReferring Expression SegmentationReferring Video Object SegmentationRetrievalSemantic SegmentationVideo Instance SegmentationVisual Object TrackingVisual TrackingZero Shot SegmentationSimilar Papers 제목 키워드 기반
UniFS: Universal Few-shot Instance Perception with Point Representations
Instance perception tasks (object detection, instance segmentation, pose estimation, counting) play a key role in industrial applications of visual models. As supervised learning methods suffer from high labeling cost, f…
Few-Shot LearningFew-Shot Object DetectionInstance Segmentationobject-detection+3Unifying Visual Perception by Dispersible Points Learning
We present a conceptually simple, flexible, and universal visual perception head for variant visual tasks, e.g., classification, object detection, instance segmentation and pose estimation, and different frameworks, such…
Instance SegmentationObjectobject-detectionObject Detection+3Aligning and Prompting Everything All at Once for Universal Visual Perception
Vision foundation models have been explored recently to build general-purpose vision systems. However, predominant paradigms, driven by casting instance-level tasks as an object-word alignment, bring heavy cross-modality…
AllObjectobject-detectionObject Detection+5Three-in-One: Robust Enhanced Universal Transferable Anti-Facial Retrieval in Online Social Networks
Deep hash-based retrieval techniques are widely used in facial retrieval systems to improve the efficiency of facial matching. However, it also carries the danger of exposing private information. Deep hash models are eas…
Meta-LearningRetrievalV-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
Multimodal Large Language Models (MLLMs) have recently been applied to universal multimodal retrieval, where Chain-of-Thought (CoT) reasoning improves candidate reranking. However, existing approaches remain largely lang…
Reinforcement Learning