PatchNet -- Short-range Template Matching for Efficient Video Processing
Object recognition is a fundamental problem in many video processing tasks, accurately locating seen objects at low computation cost paves the way for on-device video recognition. We propose PatchNet, an efficient convolutional neural network to match objects in adjacent video frames. It learns the patchwise correlation features instead of pixel features. PatchNet is very compact, running at just 58MFLOPs, $5\times$ simpler than MobileNetV2. We demonstrate its application on two tasks, video object detection and visual object tracking. On ImageNet VID, PatchNet reduces the flops of R-FCN ResNet-101 by 5x and EfficientDet-D0 by 3.4x with less than 1% mAP loss. On OTB2015, PatchNet reduces SiamFC and SiamRPN by 2.5x with no accuracy loss. Experiments on Jetson Nano further demonstrate 2.8x to 4.3x speed-ups associated with flops reduction. Code is open sourced at https://github.com/RalphMao/PatchNet.
Code (1)
Tasks
Objectobject-detectionObject DetectionObject RecognitionObject TrackingTemplate MatchingVideo Object DetectionVideo RecognitionVisual Object TrackingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PatchNet: A Tool for Deep Patch Classification
This work proposes PatchNet, an automated tool based on hierarchical deep learning for classifying patches by extracting features from commit messages and code changes. PatchNet contains a deep hierarchical structure tha…
ClassificationDeep LearningGeneral ClassificationMedicalPatchNet: A Patch-Based Self-Explainable AI Architecture for Chest X-ray Classification
Deep neural networks excel in radiological image classification but frequently suffer from poor interpretability, limiting clinical acceptance. We present MedicalPatchNet, an inherently self-explainable architecture for …
Image ClassificationTTVOS: Lightweight Video Object Segmentation with Adaptive Template Attention Module and Temporal Consistency Loss
Semi-supervised video object segmentation (semi-VOS) is widely used in many applications. This task is tracking class-agnostic objects from a given target mask. For doing this, various approaches have been developed base…
ObjectObject LocalizationOptical Flow EstimationSemantic Segmentation+4Deep-VFX: Deep Action Recognition Driven VFX for Short Video
Human motion is a key function to communicate information. In the application, short-form mobile video is so popular all over the world such as Tik Tok. The users would like to add more VFX so as to pursue creativity and…
Action RecognitionRhythmTemplate MatchingPatchnet: Interpretable Neural Networks for Image Classification
Understanding how a complex machine learning model makes a classification decision is essential for its acceptance in sensitive areas such as health care. Towards this end, we present PatchNet, a method that provides the…
BIG-bench Machine LearningClassificationGeneral Classificationimage-classification+1