Data-efficient Neural Text Compression with Interactive Learning
Neural sequence-to-sequence models have been successfully applied to text compression. However, these models were trained on huge automatically induced parallel corpora, which are only available for a few domains and tasks. In this paper, we propose a novel interactive setup to neural text compression that enables transferring a model to new domains and compression tasks with minimal human supervision. This is achieved by employing active learning, which intelligently samples from a large pool of unlabeled data. Using this setup, we can successfully adapt a model trained on small data of 40k samples for a headline generation task to a general text compression dataset at an acceptable compression quality with just 500 sampled instances annotated by a human.
Code (1)
Tasks
Active LearningHeadline GenerationSmall Data Image ClassificationText CompressionSimilar Papers 제목 키워드 기반
Comp-X: On Defining an Interactive Learned Image Compression Paradigm With Expert-driven LLM Agent
We present Comp-X, the first intelligently interactive image compression paradigm empowered by the impressive reasoning capability of large language model (LLM) agent. Notably, commonly used image codecs usually suffer f…
Image CompressionHaiku to Opus in Just 10 bits: LLMs Unlock Large Compression Gains
We study the compression of LLM-generated text across lossless and lossy regimes, characterizing a compression-compute frontier where more compression is possible at the cost of more compute. For lossless compression, do…
Fine-Grained Motion Compression and Selective Temporal Fusion for Neural B-Frame Video Coding
With the remarkable progress in neural P-frame video coding, neural B-frame coding has recently emerged as a critical research direction. However, most existing neural B-frame codecs directly adopt P-frame coding tools w…
An Interactive Annotation Tool for Perceptual Video Compression
Human perception is at the core of lossy video compression and yet, it is challenging to collect data that is sufficiently dense to drive compression. In perceptual quality assessment, human feedback is typically collect…
Video CompressionNemotron-Labs-3-Puzzle-75B-A9B: Compressing Hybrid MoE LLMs
We present Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super optimized for interactive deployment. We designed the model to maximize server throughput under high user throughput constraints. In int…
Knowledge DistillationReinforcement Learning