Streaming-capable High-performance Architecture of Learned Image Compression Codecs
Learned image compression allows achieving state-of-the-art accuracy and compression ratios, but their relatively slow runtime performance limits their usage. While previous attempts on optimizing learned image codecs focused more on the neural model and entropy coding, we present an alternative method to improving the runtime performance of various learned image compression models. We introduce multi-threaded pipelining and an optimized memory model to enable GPU and CPU workloads asynchronous execution, fully taking advantage of computational resources. Our architecture alone already produces excellent performance without any change to the neural model itself. We also demonstrate that combining our architecture with previous tweaks to the neural models can further improve runtime performance. We show that our implementations excel in throughput and latency compared to the baseline and demonstrate the performance of our implementations by creating a real-time video streaming encoder-decoder sample application, with the encoder running on an embedded device.
Code (1)
Tasks
CPUDecoderGPUImage CompressionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement
We propose a first streaming accent conversion (AC) model that transforms non-native speech into a native-like accent while preserving speaker identity, prosody and improving pronunciation. Our approach enables stream pr…
text-to-speechText to SpeechHardware-Efficient Deconvolution-Based GAN for Edge Computing
Generative Adversarial Networks (GAN) are cutting-edge algorithms for generating new data samples based on the learned data distribution. However, its performance comes at a significant cost in terms of computation and m…
Edge-computingHigh-Level SynthesisStreaming Align-Refine for Non-autoregressive Deliberation
We propose a streaming non-autoregressive (non-AR) decoding algorithm to deliberate the hypothesis alignment of a streaming RNN-T model. Our algorithm facilitates a simple greedy decoding procedure, and at the same time …
DecoderUltra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions
While recent autoregressive video diffusion models achieve remarkable streaming quality, they remain confined to low resolutions (e.g., 480P), leaving efficient, scalable, real-time high-resolution video generation a fun…
Video GenerationDecoder-only Streaming Transformer for Simultaneous Translation
Simultaneous Machine Translation (SiMT) generates translation while reading source tokens, essentially producing the target prefix based on the source prefix. To achieve good performance, it leverages the relationship be…
DecoderMachine TranslationTranslation