Entropy-Tree: Tree-Based Decoding with Entropy-Guided Exploration
Large language models achieve strong reasoning performance, yet existing decoding strategies either explore blindly (random sampling) or redundantly (independent multi-sampling). We propose Entropy-Tree, a tree-based decoding method that exploits entropy as a signal for branching decisions--expanding the search tree only at positions where the model exhibits genuine uncertainty. Entropy-Tree shows superior accuracy and calibration in reasoning tasks: it achieves better pass@k than Multi-chain across multiple models and datasets, and its predictive entropy demonstrates better AUROC compared to several traditional metrics. Entropy-Tree unifies efficient structured exploration and reliable uncertainty estimation within a single decoding procedure.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
SAGE: Accelerating Vision-Language Models via Entropy-Guided Adaptive Speculative Decoding
Speculative decoding has emerged as a promising approach to accelerate inference in vision-language models (VLMs) by enabling parallel verification of multiple draft tokens. However, existing methods rely on static tree …
EntMTP: Accelerating LLM Inference with Entropy Guided Multi Token Prediction
Multi-token prediction has been shown to increase data density during training, improve downstream text-generation quality, and serves as the defacto approach for self-speculative decoding. Existing foundation and open s…
Multiscale Latent-Guided Entropy Model for LiDAR Point Cloud Compression
The non-uniform distribution and extremely sparse nature of the LiDAR point cloud (LPC) bring significant challenges to its high-efficient compression. This paper proposes a novel end-to-end, fully-factorized deep framew…
Point Cloud Compression with Sibling Context and Surface Priors
We present a novel octree-based multi-level framework for large-scale point cloud compression, which can organize sparse and unstructured point clouds in a memory-efficient way. In this framework, we propose a new entrop…
Acceptance Dynamics Across Cognitive Domains in Speculative Decoding
Speculative decoding accelerates large language model (LLM) inference. It uses a small draft model to propose a tree of future tokens. A larger target model then verifies these tokens in a single batched forward pass. De…
Mathematical ReasoningLogical ReasoningCode Generation