DenseCap: Fully Convolutional Localization Networks for Dense Captioning

2015-11-24 · CVPR 2016 6 · Justin Johnson, Andrej Karpathy, Li Fei-Fei

We introduce the dense captioning task, which requires a computer vision system to both localize and describe salient regions in images in natural language. The dense captioning task generalizes object detection when the descriptions consist of a single word, and Image Captioning when one predicted region covers the full image. To address the localization and description task jointly we propose a Fully Convolutional Localization Network (FCLN) architecture that processes an image with a single, efficient forward pass, requires no external regions proposals, and can be trained end-to-end with a single round of optimization. The architecture is composed of a Convolutional Network, a novel dense localization layer, and Recurrent Neural Network language model that generates the label sequences. We evaluate our network on the Visual Genome dataset, which comprises 94,000 images and 4,100,000 region-grounded captions. We observe both speed and accuracy improvements over baselines based on current state of the art approaches in both generation and retrieval settings.

📄 PDF Abstract BibTeX arXiv:1511.07571

Code (1)

jcjohnson/densecap 공식 구현 torch

Tasks

Dense Captioning Image CaptioningLanguage ModelingLanguage Modelling object-detection Object Detection Retrieval

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Deep image representations using caption generators

2017-05-25 · Konda Reddy Mopuri, Vishal B. Athreya, R. Venkatesh Babu

Deep learning exploits large volumes of labeled data to learn powerful models. When the target dataset is small, it is a common practice to perform transfer learning using pre-trained models to learn new task specific re…

Retrieval Transfer Learning

DenseStep2M: A Scalable, Training-Free Pipeline for Dense Instructional Video Annotation

2026-04-29 · Mingji Ge, Qirui Chen, Zeqian Li, Weidi Xie arxiv

Long-term video understanding requires interpreting complex temporal events and reasoning over procedural activities. While instructional video corpora, like HowTo100M, offer rich resources for model training, they prese…

Zero-shot Generalization Dense Video Captioning Cross-Modal Retrieval

3M: Multi-style image caption generation using Multi-modality features under Multi-UPDOWN model

2021-03-20 · Chengxi Li, Brent Harrison

In this paper, we build a multi-style generative model for stylish image captioning which uses multi-modality image features, ResNeXt features and text features generated by DenseCap. We propose the 3M model, a Multi-UPD…

Caption Generation Image Captioning

Where to Play: Retrieval of Video Segments using Natural-Language Queries

2017-07-02 · Sangkuk Lee, Daesik Kim, Myunggi Lee, Jihye Hwang 외

In this paper, we propose a new approach for retrieval of video segments using natural language queries. Unlike most previous approaches such as concept-based methods or rule-based structured models, the proposed method …

Image Captioning Natural Language Queries Retrieval Text Generation

Weakly Supervised Dense Event Captioning in Videos

2018-12-10 · NeurIPS 2018 12 · Xuguang Duan, Wenbing Huang, Chuang Gan, Jingdong Wang 외

Dense event captioning aims to detect and describe all events of interest contained in a video. Despite the advanced development in this area, existing methods tackle this task by making use of dense temporal annotations…

Sentence