MMG-Vid: Maximizing Marginal Gains at Segment-level and Token-level for Efficient Video LLMs

2025-08-28 · Junpeng Ma, Qizhe Zhang, Ming Lu, Zhibin Wang, Qiang Zhou, Jun Song, Shanghang Zhang arxiv

Video Large Language Models (VLLMs) excel in video understanding, but their excessive visual tokens pose a significant computational challenge for real-world applications. Current methods aim to enhance inference efficiency by visual token pruning. However, they do not consider the dynamic characteristics and temporal dependencies of video frames, as they perceive video understanding as a multi-frame task. To address these challenges, we propose MMG-Vid, a novel training-free visual token pruning framework that removes redundancy by Maximizing Marginal Gains at both segment-level and token-level. Specifically, we first divide the video into segments based on frame similarity, and then dynamically allocate the token budget for each segment to maximize the marginal gain of each segment. Subsequently, we propose a temporal-guided DPC algorithm that jointly models inter-frame uniqueness and intra-frame diversity, thereby maximizing the marginal gain of each token. By combining both stages, MMG-Vid can maximize the utilization of the limited token budget, significantly improving efficiency while maintaining strong performance. Extensive experiments demonstrate that MMG-Vid can maintain over 99.5% of the original performance, while effectively reducing 75% visual tokens and accelerating the prefilling stage by 3.9x on LLaVA-OneVision-7B. Code will be released soon.

📄 PDF Abstract BibTeX arXiv:2508.21044

Code (0)

등록된 구현이 없습니다.

SMART: When is it Actually Worth Expanding a Speculative Tree?

2026-04-09 · Lifu Wang, Pan Zhou arxiv

Tree-based speculative decoding accelerates autoregressive generation by verifying a branching tree of draft tokens in a single target-model forward pass. However, existing methods prioritize maximizing token-level likel…

Quantifying Infra-Marginality and Its Trade-off with Group Fairness

2019-09-03 · Arpita Biswas, Siddharth Barman, Amit Deshpande, Amit Sharma

In critical decision-making scenarios, optimizing accuracy can lead to a biased classifier, hence past work recommends enforcing group-based fairness metrics in addition to maximizing accuracy. However, doing so exposes …

Decision Making Fairness

Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perception

2026-08-02 · Xinheng Han, Jianfei Wang, Yu Chen, Xiang Wang 외 arxiv

Multimodal Large Language Models (MLLMs) are increasingly expected to solve structured perception tasks that require visual recognition, language-to-object binding, object cardinality preservation, and precisely localize…

Reinforcement Learning

Incorporating Feature Pyramid Tokenization and Open Vocabulary Semantic Segmentation

2024-12-18 · Jianyu Zhang, Li Zhang, Shijian Li

The visual understanding are often approached from 3 granular levels: image, patch and pixel. Visual Tokenization, trained by self-supervised reconstructive learning, compresses visual data by codebook in patch-level wit…

Open Vocabulary Semantic Segmentation Open-Vocabulary Semantic Segmentation Segmentation Semantic Segmentation

Owen-Shapley Policy Optimization: A Principled RL Algorithm for Generative Search LLMs

2026-01-13 · Abhijnan Nath, Alireza Bagheri Garakani, Tianchen Zhou, Fan Yang 외 arxiv

Large language models are increasingly trained via reinforcement learning for personalized recommendation tasks, but standard methods like GRPO rely on sparse, sequence-level rewards. These obscure which tokens actually …