Query-guided Regression Network with Context Policy for Phrase Grounding
Given a textual description of an image, phrase grounding localizes objects in the image referred by query phrases in the description. State-of-the-art methods address the problem by ranking a set of proposals based on the relevance to each query, which are limited by the performance of independent proposal generation systems and ignore useful cues from context in the description. In this paper, we adopt a spatial regression method to break the performance limit, and introduce reinforcement learning techniques to further leverage semantic context information. We propose a novel Query-guided Regression network with Context policy (QRC Net) which jointly learns a Proposal Generation Network (PGN), a Query-guided Regression Network (QRN) and a Context Policy Network (CPN). Experiments show QRC Net provides a significant improvement in accuracy on two popular datasets: Flickr30K Entities and Referit Game, with 14.25% and 17.14% increase over the state-of-the-arts respectively.
Code (0)
등록된 구현이 없습니다.
Tasks
Phrase GroundingregressionReinforcement LearningSimilar Papers 제목 키워드 기반
Position-aware Location Regression Network for Temporal Video Grounding
The key to successful grounding for video surveillance is to understand a semantic phrase corresponding to important actors and objects. Conventional methods ignore comprehensive contexts for the phrase or require heavy …
PositionregressionVideo GroundingOptimizing Test-Time Query Representations for Dense Retrieval
Recent developments of dense retrieval rely on quality representations of queries and contexts from pre-trained query and context encoders. In this paper, we introduce TOUR (Test-Time Optimization of Query Representation…
Contrastive LearningInformation RetrievalOpen-Domain Question AnsweringPassage Retrieval+3QueryBandits for Hallucination Mitigation: Exploiting Semantic Features for No-Regret Rewriting
Advanced reasoning capabilities in Large Language Models (LLMs) have caused higher hallucination prevalence; yet most mitigation work focuses on after-the-fact filtering rather than shaping the queries that trigger them.…
Query-Based Keyphrase Extraction from Long Documents
Transformer-based architectures in natural language processing force input size limits that can be problematic when long documents need to be processed. This paper overcomes this issue for keyphrase extraction by chunkin…
ChunkingKeyphrase ExtractionDocQAC: Adaptive Trie-Guided Decoding for Effective In-Document Query Auto-Completion
Query auto-completion (QAC) has been widely studied in the context of web search, yet remains underexplored for in-document search, which we term DocQAC. DocQAC aims to enhance search productivity within long documents b…