BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection

2026-03-20 · Zhengpei Hu, Kai Li, Dapeng Fu, Chang Zeng, Yue Li, Yuanhao Tang, Jianqiang Huang arxiv

The exponential expansion of context windows in LLMs has unlocked capabilities for long-document understanding but introduced severe bottlenecks in inference latency and information utilization. Existing compression methods often suffer from high training costs or semantic fragmentation due to aggressive token pruning. In this paper, we propose BEAVER, a novel training-free framework that shifts compression from linear token removal to structure-aware hierarchical selection. BEAVER maximizes hardware parallelism by mapping variable-length contexts into dense page-level tensors via dual-path pooling, and preserves discourse integrity through a hybrid planner combining semantic and lexical dual-branch selection with sentence smoothing. Extensive evaluations on four long-context benchmarks demonstrate that BEAVER achieves comparable performance to state-of-the-art (SOTA) methods like LongLLMLingua. Notably, on the RULER benchmark, BEAVER maintains high fidelity in multi-needle retrieval where baselines deteriorate. Regarding efficiency, BEAVER reduces latency by 26.4x on 128k contexts, offering a scalable solution for high-throughput applications. Our code is available at https://cslikai.cn/BEAVER/.

📄 PDF Abstract BibTeX arXiv:2603.19635

Code (0)

등록된 구현이 없습니다.

BEAVER: An Efficient Deterministic LLM Verifier

2025-12-05 · Tarun Suresh, Nalin Wadhwa, Debangshu Banerjee, Gagandeep Singh arxiv

As large language models (LLMs) transition from research prototypes to production systems, practitioners often need reliable methods to verify model outputs and characterize tail risk for safe deployment. While sampling-…

Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression

2026-08-05 · Zhengpei Hu, Kai Li, Dapeng Fu, Xuechao Zou 외 hf

Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure…

Multi-hop Question Answering

Adaptive Targeted Dynamic Chunking for Tokenization-Free Hierarchical Model

2026-05-28 · Thang Dang, Akira Nakagawa, Kenichi Kobayashi, Koichi Shirahata arxiv

Tokenization-free hierarchical models are emerging as a promising alternative to traditional Large Language Models (LLMs), addressing inherent preprocessing issues such as vocabulary design complexity, out-of-vocabulary …

BEAVER: An Enterprise Benchmark for Text-to-SQL

2024-09-03 · Peter Baile Chen, Fabian Wenz, Yi Zhang, Devin Yang 외

Existing text-to-SQL benchmarks have largely been constructed from web tables with human-generated question-SQL pairs. LLMs typically show strong results on these benchmarks, leading to a belief that LLMs are effective a…

Natural Language Queries Prompt Engineering RAG Text to SQL+1

Dynamic Compressing Prompts for Efficient Inference of Large Language Models

2025-04-15 · Jinwu Hu, Wei zhang, Yufeng Wang, Yu Hu 외

Large Language Models (LLMs) have shown outstanding performance across a variety of tasks, partly due to advanced prompting techniques. However, these techniques often require lengthy prompts, which increase computationa…

Token Reduction