Papers Cross-Modal Information Retrieval
“Cross-Modal Information Retrieval” 태그가 달린 논문 16편 · 필터 해제
Multimodal Representation Alignment for Cross-modal Information Retrieval
Different machine learning models can represent the same underlying concept in different ways. This variability is particularly valuable for in-the-wild multimodal retrieval, where the objective is to identify the corres…
Cross-Modal Information RetrievalInformation RetrievalRetrievalSentencejina-clip-v2: Multilingual Multimodal Embeddings for Text and Images
Contrastive Language-Image Pretraining (CLIP) has been widely used for crossmodal information retrieval and multimodal understanding tasks. However, CLIP models are mainly optimized for crossmodal vision-language tasks a…
Contrastive LearningCross-Modal Information RetrievalInformation RetrievalRepresentation Learning+3Is Cross-modal Information Retrieval Possible without Training?
Encoded representations from a pretrained deep learning model (e.g., BERT text embeddings, penultimate CNN layer activations of an image) convey a rich set of features beneficial for information retrieval. Embeddings for…
Contrastive LearningCross-Modal Information RetrievalImage to textImage-to-Text Retrieval+3Multi-modal Machine Learning in Engineering Design: A Review and Future Directions
In the rapidly advancing field of multi-modal machine learning (MMML), the convergence of multiple data modalities has the potential to reshape various applications. This paper presents a comprehensive overview of the cu…
Cross-Modal Information RetrievalDesign SynthesisInformation RetrievalRetrievalVITR: Augmenting Vision Transformers with Relation-Focused Learning for Cross-Modal Information Retrieval
The relations expressed in user queries are vital for cross-modal information retrieval. Relation-focused cross-modal retrieval aims to retrieve information that corresponds to these relations, enabling effective retriev…
Cross-Modal Information RetrievalCross-Modal RetrievalImage RetrievalImage to text+3Improving Visual-Semantic Embeddings by Learning Semantically-Enhanced Hard Negatives for Cross-modal Information Retrieval
Visual Semantic Embedding (VSE) aims to extract the semantics of images and their descriptions, and embed them into the same latent space for cross-modal information retrieval. Most existing VSE networks are trained by a…
Cross-Modal Information RetrievalImage DescriptionInformation RetrievalRetrievalLILE: Look In-Depth before Looking Elsewhere -- A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archives
The volume of available data has grown dramatically in recent years in many applications. Furthermore, the age of networks that used multiple modalities separately has practically ended. Therefore, enabling bidirectional…
Cross-Modal Information RetrievalCross-Modal RetrievalInformation RetrievalRetrievalVisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words
Text-to-image retrieval is an essential task in cross-modal information retrieval, i.e., retrieving relevant images from a large and unlabelled dataset given textual queries. In this paper, we propose VisualSparta, a nov…
CPUCross-Modal Information RetrievalCross-Modal RetrievalGPU+3Learning the Best Pooling Strategy for Visual Semantic Embedding
Visual Semantic Embedding (VSE) is a dominant approach for vision-language retrieval, which aims at learning a deep embedding space such that visual data are embedded close to their semantic text labels or descriptions. …
Cross-Modal Information RetrievalImage-text RetrievalRetrievalSemantic Image-Text Similarity+2Fine-grained Visual Textual Alignment for Cross-Modal Retrieval using Transformer Encoders
Despite the evolution of deep-learning-based visual-textual processing systems, precise multi-modal matching remains a challenging task. In this work, we tackle the task of cross-modal retrieval through image-sentence ma…
Cross-Modal Information RetrievalCross-Modal RetrievalImage RetrievalInformation Retrieval+3ZSCRGAN: A GAN-based Expectation Maximization Model for Zero-Shot Retrieval of Images from Textual Descriptions
Most existing algorithms for cross-modal Information Retrieval are based on a supervised train-test setup, where a model learns to align the mode of the query (e.g., text) to the mode of the documents (e.g., images) from…
Cross-Modal Information RetrievalImage RetrievalInformation RetrievalRetrieval+3Cross-modal representation alignment of molecular structure and perturbation-induced transcriptional profiles
Modeling the relationship between chemical structure and molecular activity is a key goal in drug development. Many benchmark tasks have been proposed for molecular property prediction, but these tasks are generally aime…
Cross-Modal Information RetrievalInformation RetrievalMolecular Property PredictionProperty Prediction+2CMIR-NET : A Deep Learning Based Model For Cross-Modal Retrieval In Remote Sensing
We address the problem of cross-modal information retrieval in the domain of remote sensing. In particular, we are interested in two application scenarios: i) cross-modal retrieval between panchromatic (PAN) and multi-sp…
Cross-Modal Information RetrievalCross-Modal RetrievalImage RetrievalInformation Retrieval+2Scene Graph Reasoning with Prior Visual Relationship for Visual Question Answering
One of the key issues of Visual Question Answering (VQA) is to reason with semantic clues in the visual content under the guidance of the question, how to model relational semantics still remains as a great challenge. To…
Cross-Modal Information RetrievalInformation RetrievalQuestion AnsweringVisual Question Answering+1Modeling Text with Graph Convolutional Network for Cross-Modal Information Retrieval
Cross-modal information retrieval aims to find heterogeneous data of various modalities from a given query of one modality. The main challenge is to map different modalities into a common semantic space, in which distanc…
Cross-Modal Information RetrievalInformation RetrievalRetrievalPicture It In Your Mind: Generating High Level Visual Representations From Textual Descriptions
In this paper we tackle the problem of image search when the query is a short textual description of the image the user is looking for. We choose to implement the actual search process as a similarity search in a visual …
Cross-Modal Information RetrievalCross-Modal RetrievalDescriptiveImage Retrieval+1