Fixed-Point Convolutional Neural Network for Real-Time Video Processing in FPGA
Modern mobile neural networks with a reduced number of weights and parameters do a good job with image classification tasks, but even they may be too complex to be implemented in an FPGA for video processing tasks. The article proposes neural network architecture for the practical task of recognizing images from a camera, which has several advantages in terms of speed. This is achieved by reducing the number of weights, moving from a floating-point to a fixed-point arithmetic, and due to a number of hardware-level optimizations associated with storing weights in blocks, a shift register, and an adjustable number of convolutional blocks that work in parallel. The article also proposed methods for adapting the existing data set for solving a different task. As the experiments showed, the proposed neural network copes well with real-time video processing even on the cheap FPGAs.
Code (2)
Tasks
image-classificationImage ClassificationSimilar Papers 제목 키워드 기반
FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers
Real-time video generation demands fast decoding as much as fast denoising, yet current latent video diffusion models rely on 3D convolutional decoders that are slow and memory-intensive at high resolutions or for long v…
Video GenerationAction Keypoint Network for Efficient Video Recognition
Reducing redundancy is crucial for improving the efficiency of video recognition models. An effective approach is to select informative content from the holistic video, yielding a popular family of dynamic video recognit…
Action RecognitionPoint Cloud ClassificationVideo RecognitionRigPAPR: Rig-Based Animation of Static Neural Point Clouds from a Fixed-Viewpoint Video
Static neural point reconstructions capture a subject at high fidelity from posed images. Given such a reconstruction, we aim to animate it to follow a monocular fixed-viewpoint driving video of the subject, whether capt…
Point CloudsVid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation
Video generation models often operate under the assumption of fixed frame rates, which leads to suboptimal performance when it comes to handling flexible frame rates (e.g., increasing the frame rate of the more dynamic p…
DecoderVideo Generation4DSR-GCN: 4D Video Point Cloud Upsampling using Graph Convolutional Networks
Time varying sequences of 3D point clouds, or 4D point clouds, are now being acquired at an increasing pace in several applications (e.g., LiDAR in autonomous or assisted driving). In many cases, such volume of data is t…
Edge-computingGraph Attentionpoint cloud upsampling