Multitask Text-to-Visual Embedding with Titles and Clickthrough Data
Text-visual (or called semantic-visual) embedding is a central problem in vision-language research. It typically involves mapping of an image and a text description to a common feature space through a CNN image encoder and a RNN language encoder. In this paper, we propose a new method for learning text-visual embedding using both image titles and click-through data from an image search engine. We also propose a new triplet loss function by modeling positive awareness of the embedding, and introduce a novel mini-batch-based hard negative sampling approach for better data efficiency in the learning process. Experimental results show that our proposed method outperforms existing methods, and is also effective for real-world text-to-visual retrieval.
Code (0)
등록된 구현이 없습니다.
Tasks
Image RetrievalRetrievalTripletMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Continuous Prompt Tuning Based Textual Entailment Model for E-commerce Entity Typing
The explosion of e-commerce has caused the need for processing and analysis of product titles, like entity typing in product titles. However, the rapid activity in e-commerce has led to the rapid emergence of new entitie…
Entity TypingNatural Language InferenceHooks in the Headline: Learning to Generate Headlines with Controlled Styles
Current summarization systems only produce plain, factual headlines, but do not meet the practical needs of creating memorable titles to increase exposure. We propose a new task, Stylistic Headline Generation (SHG), to e…
Headline GenerationJAMES: Normalizing Job Titles with Multi-Aspect Graph Embeddings and Reasoning
In online job marketplaces, it is important to establish a well-defined job title taxonomy for various downstream tasks (e.g., job recommendation, users' career analysis, and turnover prediction). Job Title Normalization…
Logical ReasoningSemantic SimilaritySemantic Textual SimilarityMultitasking Embedding for Embryo Blastocyst Grading Prediction (MEmEBG)
Reliable evaluation of blastocyst quality is critical for the success of in vitro fertilization (IVF) treatments. Current embryo grading practices primarily rely on visual assessment of morphological features, which intr…
Document Embedding for Scientific Articles: Efficacy of Word Embeddings vs TFIDF
Over the last few years, neural network derived word embeddings became popular in the natural language processing literature. Studies conducted have mostly focused on the quality and application of word embeddings traine…
ArticlesDocument EmbeddingWord Embeddings