Relation-Aware Distribution Representation Network for Person Clustering with Multiple Modalities
Person clustering with multi-modal clues, including faces, bodies, and voices, is critical for various tasks, such as movie parsing and identity-based movie editing. Related methods such as multi-view clustering mainly project multi-modal features into a joint feature space. However, multi-modal clue features are usually rather weakly correlated due to the semantic gap from the modality-specific uniqueness. As a result, these methods are not suitable for person clustering. In this paper, we propose a Relation-Aware Distribution representation Network (RAD-Net) to generate a distribution representation for multi-modal clues. The distribution representation of a clue is a vector consisting of the relation between this clue and all other clues from all modalities, thus being modality agnostic and good for person clustering. Accordingly, we introduce a graph-based method to construct distribution representation and employ a cyclic update policy to refine distribution representation progressively. Our method achieves substantial improvements of +6% and +8.2% in F-score on the Video Person-Clustering Dataset (VPCD) and VoxCeleb2 multi-view clustering dataset, respectively. Codes will be released publicly upon acceptance.
Code (0)
등록된 구현이 없습니다.
Tasks
ClusteringRelationSimilar Papers 제목 키워드 기반
Distribution Context Aware Loss for Person Re-identification
To learn the optimal similarity function between probe and gallery images in Person re-identification, effective deep metric learning methods have been extensively explored to obtain discriminative feature embedding. How…
ClusteringMetric LearningPerson Re-IdentificationTripletRelation-Aware Global Attention for Person Re-identification
For person re-identification (re-id), attention mechanisms have become attractive as they aim at strengthening discriminative features and suppressing irrelevant ones, which matches well the key of re-id, i.e., discrimin…
ClusteringImage ClassificationPerson Re-IdentificationRelation+1Oracle-guided Contrastive Clustering
Deep clustering aims to learn a clustering representation through deep architectures. Most of the existing methods usually conduct clustering with the unique goal of maximizing clustering performance, that ignores the pe…
Active LearningClusteringContrastive LearningDeep ClusteringDeep Learning-Based Approach for Improving Relational Aggregated Search
Due to an information explosion on the internet, there is a need for the development of aggregated search systems that can boost the retrieval and management of content in various formats. To further improve the clusteri…
Representation LearningAgent-Centric Personalized Multiple Clustering with Multi-Modal LLMs
Personalized multiple clustering aims to generate diverse partitions of a dataset based on different user-specific aspects, rather than a single clustering. It has recently drawn research interest for accommodating varyi…
Clustering