Dynamic Appearance: A Video Representation for Action Recognition with Joint Training

2022-11-23 · Guoxi Huang, Adrian G. Bors

Static appearance of video may impede the ability of a deep neural network to learn motion-relevant features in video action recognition. In this paper, we introduce a new concept, Dynamic Appearance (DA), summarizing the appearance information relating to movement in a video while filtering out the static information considered unrelated to motion. We consider distilling the dynamic appearance from raw video data as a means of efficient video understanding. To this end, we propose the Pixel-Wise Temporal Projection (PWTP), which projects the static appearance of a video into a subspace within its original vector space, while the dynamic appearance is encoded in the projection residual describing a special motion pattern. Moreover, we integrate the PWTP module with a CNN or Transformer into an end-to-end training framework, which is optimized by utilizing multi-objective optimization algorithms. We provide extensive experimental results on four action recognition benchmarks: Kinetics400, Something-Something V1, UCF101 and HMDB51.

📄 PDF Abstract BibTeX arXiv:2211.12748

Code (0)

등록된 구현이 없습니다.

Tasks

Action Recognition Temporal Action LocalizationVideo Understanding

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음

Attention 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Adam 설명 없음

Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

View-invariant action recognition

2020-09-01 · Yogesh S Rawat, Shruti Vyas

Human action recognition is an important problem in computer vision. It has a wide range of applications in surveillance, human-computer interaction, augmented reality, video indexing, and retrieval. The varying pattern …

Action Recognition Retrieval Temporal Action Localization

Slicing Convolutional Neural Network for Crowd Video Understanding

2016-06-01 · CVPR 2016 6 · Jing Shao, Chen-Change Loy, Kai Kang, Xiaogang Wang

Learning and capturing both appearance and dynamic representations are pivotal for crowd video understanding. Convolutional Neural Networks (CNNs) have shown its remarkable potential in learning appearance representation…

Attribute Video Understanding

Object-Region Video Transformers

2021-10-13 · CVPR 2022 1 · Roei Herzig, Elad Ben-Avraham, Karttikeya Mangalam, Amir Bar 외

Recently, video transformers have shown great success in video understanding, exceeding CNN performance; yet existing video transformer models do not explicitly model objects, although objects can be essential for recogn…

Action Detection Action Recognition Few-Shot action recognition Few Shot Action Recognition+2

Action Recognition with Dynamic Image Networks

2016-12-02 · Hakan Bilen, Basura Fernando, Efstratios Gavves, Andrea Vedaldi

We introduce the concept of "dynamic image", a novel compact representation of videos useful for video analysis, particularly in combination with convolutional neural networks (CNNs). A dynamic image encodes temporal dat…

Action Recognition Optical Flow Estimation Temporal Action Localization

3DV: 3D Dynamic Voxel for Action Recognition in Depth Video

2020-05-12 · CVPR 2020 6 · Yancheng Wang, Yang Xiao, Fu Xiong, Wenxiang Jiang 외

To facilitate depth-based 3D action recognition, 3D dynamic voxel (3DV) is proposed as a novel 3D motion representation. With 3D space voxelization, the key idea of 3DV is to encode 3D motion information within depth vid…

3D Action Recognition Action Recognition