An Unsupervised Sampling Approach for Image-Sentence Matching Using Document-Level Structural Information
In this paper, we focus on the problem of unsupervised image-sentence matching. Existing research explores to utilize document-level structural information to sample positive and negative instances for model training. Although the approach achieves positive results, it introduces a sampling bias and fails to distinguish instances with high semantic similarity. To alleviate the bias, we propose a new sampling strategy to select additional intra-document image-sentence pairs as positive or negative samples. Furthermore, to recognize the complex pattern in intra-document samples, we propose a Transformer based model to capture fine-grained features and implicitly construct a graph for each document, where concepts in a document are introduced to bridge the representation learning of images and sentences in the context of a document. Experimental results show the effectiveness of our approach to alleviate the bias and learn well-aligned multimodal representations.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningSemantic SimilaritySemantic Textual SimilaritySentenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Centrality Meets Centroid: A Graph-based Approach for Unsupervised Document Summarization
Unsupervised document summarization has re-acquired lots of attention in recent years thanks to its simplicity and data independence. In this paper, we propose a graph-based unsupervised approach for extractive document …
Document SummarizationExtractive Document SummarizationSentenceZero-Shot Text Matching for Automated Auditing using Sentence Transformers
Natural language processing methods have several applications in automated auditing, including document or passage classification, information retrieval, and question answering. However, training such models requires a l…
Information RetrievalQuestion AnsweringRetrievalSemantic Similarity+3Combining Word Embeddings and N-grams for Unsupervised Document Summarization
Graph-based extractive document summarization relies on the quality of the sentence similarity graph. Bag-of-words or tf-idf based sentence similarity uses exact word matching, but fails to measure the semantic similarit…
DiversityDocument SummarizationExtractive Document SummarizationExtractive Summarization+8Unsupervised Document Embedding via Contrastive Augmentation
We present a contrasting learning approach with data augmentation techniques to learn document representations in an unsupervised manner. Inspired by recent contrastive self-supervised learning algorithms used for image …
Contrastive LearningData AugmentationDocument ClassificationDocument Embedding+3Optimal Partial Transport Based Sentence Selection for Long-form Document Matching
One typical approach to long-form document matching is first conducting alignment between cross-document sentence pairs, and then aggregating all of the sentence-level matching signals. However, this approach could be pr…
FormSentence