Generalized Multi-view Embedding for Visual Recognition and Cross-modal Retrieval
In this paper, the problem of multi-view embedding from different visual cues and modalities is considered. We propose a unified solution for subspace learning methods using the Rayleigh quotient, which is extensible for multiple views, supervised learning, and non-linear embeddings. Numerous methods including Canonical Correlation Analysis, Partial Least Sqaure regression and Linear Discriminant Analysis are studied using specific intrinsic and penalty graphs within the same framework. Non-linear extensions based on kernels and (deep) neural networks are derived, achieving better performance than the linear ones. Moreover, a novel Multi-view Modular Discriminant Analysis (MvMDA) is proposed by taking the view difference into consideration. We demonstrate the effectiveness of the proposed multi-view embedding methods on visual object recognition and cross-modal image retrieval, and obtain superior results in both applications compared to related methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Cross-Modal RetrievalImage RetrievalObject RecognitionRetrievalSimilar Papers 제목 키워드 기반
Audio-Visual Speech Enhancement and Separation by Utilizing Multi-Modal Self-Supervised Embeddings
AV-HuBERT, a multi-modal self-supervised learning model, has been shown to be effective for categorical problems such as automatic speech recognition and lip-reading. This suggests that useful audio-visual speech represe…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Lip Readingregression+5Generalized Multi-view Shared Subspace Learning using View Bootstrapping
A key objective in multi-view learning is to model the information common to multiple parallel views of a class of objects/events to improve downstream learning tasks. In this context, two open research questions remain:…
3D Object ClassificationFace RecognitionMULTI-VIEW LEARNINGRobust Face RecognitionSpectral Graph-Based Method of Multimodal Word Embedding
In this paper, we propose a novel method for multimodal word embedding, which exploit a generalized framework of multi-view spectral graph embedding to take into account visual appearances or scenes denoted by words in a…
Graph EmbeddingImage RetrievalMachine TranslationPart-Of-Speech Tagging+4Generalized Zero-Shot Recognition based on Visually Semantic Embedding
We propose a novel Generalized Zero-Shot learning (GZSL) method that is agnostic to both unseen images and unseen semantic vectors during training. Prior works in this context propose to map high-dimensional visual featu…
AttributeGeneralized Zero-Shot LearningZero-Shot LearningMulti-view Distillation based on Multi-modal Fusion for Few-shot Action Recognition(CLIP-$\mathrm{M^2}$DF)
In recent years, few-shot action recognition has attracted increasing attention. It generally adopts the paradigm of meta-learning. In this field, overcoming the overlapping distribution of classes and outliers is still …
Action RecognitionFew-Shot action recognitionFew Shot Action RecognitionMeta-Learning