Video Instance Segmentation Tracking With a Modified VAE Architecture

2020-06-01 · CVPR 2020 6 · Chung-Ching Lin, Ying Hung, Rogerio Feris, Linglin He

We propose a modified variational autoencoder (VAE) architecture built on top of Mask R-CNN for instance-level video segmentation and tracking. The method builds a shared encoder and three parallel decoders, yielding three disjoint branches for predictions of future frames, object detection boxes, and instance segmentation masks. To effectively solve multiple learning tasks, we introduce a Gaussian Process model to enhance the statistical representation of VAE by relaxing the prior strong independent and identically distributed (iid) assumption of conventional VAEs and allowing potential correlations among extracted latent variables. The network learns embedded spatial interdependence and motion continuity in video data and creates a representation that is effective to produce high-quality segmentation masks and track multiple instances in diverse and unstructured videos. Evaluation on a variety of recently introduced datasets shows that our model outperforms previous methods and achieves the new best in class performance.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Instance Segmentation object-detection Object Detection Segmentation Semantic Segmentation Video Instance Segmentation Video Segmentation Video Semantic Segmentation

Methods 이 논문이 사용한 방법론

RPN A Region Proposal Network, or RPN, is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

RoIAlign Region of Interest Align, or RoIAlign, is an operation for extracting a small feature map from each RoI in detection and segmentation based tasks. It removes the harsh…

Mask R-CNN Mask R-CNN extends Faster R-CNN to solve instance segmentation tasks. It achieves this by adding a branch for predicting an…

Solana Customer Service Number +1-833-534-1729 설명 없음

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

USD Coin Customer Service Number +1-833-534-1729 설명 없음

Instance Segmentation and Tracking with Cosine Embeddings and Recurrent Hourglass Networks

2018-06-06 · Christian Payer, Darko Štern, Thomas Neff, Horst Bischof 외

Different to semantic segmentation, instance segmentation assigns unique labels to each individual instance of the same class. In this work, we propose a novel recurrent fully convolutional network architecture for track…

Instance Segmentation Segmentation Semantic Segmentation

Robust Online Video Instance Segmentation with Track Queries

2022-11-16 · Zitong Zhan, Daniel McKee, Svetlana Lazebnik

Recently, transformer-based methods have achieved impressive results on Video Instance Segmentation (VIS). However, most of these top-performing methods run in an offline manner by processing the entire video clip at onc…

Image Segmentation Instance Segmentation Multi-Object Tracking Object Tracking+5

Video Instance Segmentation

2019-05-12 · ICCV 2019 10 · Linjie Yang, Yuchen Fan, Ning Xu

In this paper we present a new computer vision task, named video instance segmentation. The goal of this new task is simultaneous detection, segmentation and tracking of instances in videos. In words, it is the first tim…

Instance Segmentation Segmentation Semantic Segmentation Video Instance Segmentation+1

ASIST: Annotation-free synthetic instance segmentation and tracking for microscope video analysis

2020-11-02 · Quan Liu, Isabella M. Gaeta, Mengyang Zhao, Ruining Deng 외

Instance object segmentation and tracking provide comprehensive quantification of objects across microscope videos. The recent single-stage pixel-embedding based deep learning approach has shown its superior performance …

Generative Adversarial Network Image Segmentation Instance Segmentation Segmentation+1

Foundation Models for Amodal Video Instance Segmentation in Automated Driving

2024-09-21 · Jasmin Breitenstein, Franz Jünger, Andreas Bär, Tim Fingscheidt

In this work, we study amodal video instance segmentation for automated driving. Previous works perform amodal video instance segmentation relying on methods trained on entirely labeled video data with techniques borrowe…

Amodal Instance Segmentation Instance Segmentation Point Tracking Segmentation+2