Shot Contrastive Self-Supervised Learning for Scene Boundary Detection

2021-04-28 · CVPR 2021 1 · Shixing Chen, Xiaohan Nie, David Fan, Dongqing Zhang, Vimal Bhat, Raffay Hamid

Scenes play a crucial role in breaking the storyline of movies and TV episodes into semantically cohesive parts. However, given their complex temporal structure, finding scene boundaries can be a challenging task requiring large amounts of labeled training data. To address this challenge, we present a self-supervised shot contrastive learning approach (ShotCoL) to learn a shot representation that maximizes the similarity between nearby shots compared to randomly selected shots. We show how to apply our learned shot representation for the task of scene boundary detection to offer state-of-the-art performance on the MovieNet dataset while requiring only ~25% of the training labels, using 9x fewer model parameters and offering 7x faster runtime. To assess the effectiveness of ShotCoL on novel applications of scene boundary detection, we take on the problem of finding timestamps in movies and TV episodes where video-ads can be inserted while offering a minimally disruptive viewing experience. To this end, we collected a new dataset called AdCuepoints with 3,975 movies and TV episodes, 2.2 million shots and 19,119 minimally disruptive ad cue-point labels. We present a thorough empirical analysis on this dataset demonstrating the effectiveness of ShotCoL for ad cue-points detection.

📄 PDF Abstract BibTeX arXiv:2104.13537

Code (0)

등록된 구현이 없습니다.

Tasks

Boundary Detection Contrastive Learning Self-Supervised Learning

Methods 이 논문이 사용한 방법론

Contrastive Learning 설명 없음

Boundary-aware Self-supervised Learning for Video Scene Segmentation

2022-01-14 · Jonghwan Mun, Minchul Shin, Gunsoo Han, Sangho Lee 외

Self-supervised learning has drawn attention through its effectiveness in learning in-domain representations with no ground-truth annotations; in particular, it is shown that properly designed pretext tasks (e.g., contra…

Scene Segmentation Self-Supervised Learning

Boundary-aware Pre-training for Video Scene Segmentation

2021-09-29 · Jonghwan Mun, Minchul Shin, Gunsoo Han, Sangho Lee 외

Scene Segmentation Self-Supervised Learning

Scene Consistency Representation Learning for Video Scene Segmentation

2022-05-11 · CVPR 2022 1 · Haoqian Wu, Keyu Chen, Yanan Luo, Ruizhi Qiao 외

A long-term video, such as a movie or TV show, is composed of various scenes, each of which represents a series of shots sharing the same semantic story. Spotting the correct scene boundary from the long-term video is a …

Data Augmentation Inductive Bias Representation Learning Scene Segmentation+1

SnapshotNet: Self-supervised Feature Learning for Point Cloud Data Segmentation Using Minimal Labeled Data

2022-01-13 · Xingye Li, Ling Zhang, Zhigang Zhu

Manually annotating complex scene point cloud datasets is both costly and error-prone. To reduce the reliance on labeled data, a new model called SnapshotNet is proposed as a self-supervised feature learning approach, wh…

Contrastive Learning Self-Supervised Learning Semantic Segmentation Weakly supervised segmentation

Few-Shot Image Classification via Contrastive Self-Supervised Learning

2020-08-23 · Jianyi Li, Guizhong Liu

Most previous few-shot learning algorithms are based on meta-training with fake few-shot tasks as training samples, where large labeled base classes are required. The trained model is also limited by the type of tasks. I…

Classification Few-Shot Image Classification Few-Shot Learning General Classification+5