VCR: Video representation for Contextual Retrieval

2024-02-12 · Oron Nir, Idan Vidra, Avi Neeman, Barak Kinarti, Ariel Shamir

Streamlining content discovery within media archives requires integrating advanced data representations and effective visualization techniques for clear communication of video topics to users. The proposed system addresses the challenge of efficiently navigating large video collections by exploiting a fusion of visual, audio, and textual features to accurately index and categorize video content through a text-based method. Additionally, semantic embeddings are employed to provide contextually relevant information and recommendations to users, resulting in an intuitive and engaging exploratory experience over our topics ontology map using OpenAI GPT-4.

📄 PDF Abstract BibTeX arXiv:2402.07466

Code (1)

oronnir/vcr 공식 구현

Tasks

Retrieval

Methods 이 논문이 사용한 방법론

Position-Wise Feed-Forward Layer 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Attention 설명 없음

ContextIQ: A Multimodal Expert-Based Video Retrieval System for Contextual Advertising

2024-10-29 · Ashutosh Chaubey, Anoubhav Agarwaal, Sartaki Sinha Roy, Aayush Agrawal 외

Contextual advertising serves ads that are aligned to the content that the user is viewing. The rapid growth of video content on social platforms and streaming services, along with privacy concerns, has increased the nee…

Retrieval Text to Video Retrieval Video Retrieval

CONQUER: Contextual Query-aware Ranking for Video Corpus Moment Retrieval

2021-09-21 · Zhijian Hou, Chong-Wah Ngo, Wing Kwong Chan

This paper tackles a recently proposed Video Corpus Moment Retrieval task. This task is essential because advanced video retrieval applications should enable users to retrieve a precise moment from a large video corpus. …

Corpus Video Moment Retrieval Moment Retrieval orpus Video Moment Retrieval Representation Learning+4

Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval

2025-03-24 · CVPR 2025 1 · Arun Reddy, Alexander Martin, Eugene Yang, Andrew Yates 외

In this work, we tackle the problem of text-to-video retrieval (T2VR). Inspired by the success of late interaction techniques in text-document, text-image, and text-video retrieval, our approach, Video-ColBERT, introduce…

Retrieval Text to Video Retrieval Video Retrieval

Enhancing Subsequent Video Retrieval via Vision-Language Models (VLMs)

2025-03-21 · Yicheng Duan, Xi Huang, Duo Chen

The rapid growth of video content demands efficient and precise retrieval systems. While vision-language models (VLMs) excel in representation learning, they often struggle with adaptive, time-sensitive video retrieval. …

Representation Learning Retrieval Video Retrieval

Contextual Explainable Video Representation: Human Perception-based Understanding

2022-12-12 · Khoa Vo, Kashu Yamazaki, Phong X. Nguyen, Phat Nguyen 외

Video understanding is a growing field and a subject of intense research, which includes many interesting tasks to understanding both spatial and temporal information, e.g., action detection, action recognition, video ca…

Action Detection Action Recognition Retrieval Video Captioning+2