Code Is More Than Text: Uncertainty Estimation for Code Generation

2026-06-08 · Yuling Shi, Caiqi Zhang, Yuexian Li, Haopeng Wang, Yeheng Chen, Nigel Collier, Xiaodong Gu arxiv

Large language models (LLMs) are increasingly deployed as code generators, where silently wrong programs pose real safety and reliability risks. Reliable uncertainty estimation (UE) is essential for selective prediction, human-in-the-loop review, and downstream agentic decisions. Yet most existing code UE methods are inherited from natural language (NL) generation and ignore properties that make code distinct. We argue that code differs from NL in three ways: a single wrong token can break an entire program (token fragility); algorithmic intent and concrete implementation can disagree independently (intent-code gap); and programs can be executed (executability). We instantiate these properties as three orthogonal uncertainty axes: lexical (Top-K token entropy), algorithmic (pseudo-code consistency), and functional (behavioral consistency). Across five code LLMs, our three-axis ensemble improves average AUROC from 0.696 for the strongest NL-derived baseline to 0.776 (+8.1 points). Notably, on Qwen3-14B, our single-pass Top-K token entropy matches the strongest multi-pass baseline while being over 3x cheaper; across models, it remains a competitive low-cost signal. These results suggest that code UE deserves code-specific design rather than direct NL ports.

📄 PDF Abstract BibTeX arXiv:2606.09577

Code (0)

등록된 구현이 없습니다.

Tasks

Code Generation

SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learning

2024-08-23 · Zhongjian Qiao, Jiafei Lyu, Kechen Jiao, Qi Liu 외

The performance of offline reinforcement learning (RL) suffers from the limited size and quality of static datasets. Model-based offline RL addresses this issue by generating synthetic samples through a dynamics model to…

D4RL Offline RL Reinforcement Learning (RL)

GENUINE: Graph Enhanced Multi-level Uncertainty Estimation for Large Language Models

2025-09-09 · Tuo Wang, Adithya Kulkarni, Tyler Cody, Peter A. Beling 외 arxiv

Uncertainty estimation is essential for enhancing the reliability of Large Language Models (LLMs), particularly in high-stakes applications. Existing methods often overlook semantic dependencies, relying on token-level p…

Detecting Out-of-distribution Samples via Variational Auto-encoder with Reliable Uncertainty Estimation

2020-07-16 · Xuming Ran, Mingkun Xu, Lingrui Mei, Qi Xu 외

Variational autoencoders (VAEs) are influential generative models with rich representation capabilities from the deep neural network architecture and Bayesian method. However, VAE models have a weakness that assign a hig…

Anomaly Detection

Towards Reliable Truth-Aligned Uncertainty Estimation in Large Language Models

2026-04-01 · Ponhvoan Srey, Quang Minh Nguyen, Xiaobao Wu, Anh Tuan Luu arxiv

Uncertainty estimation (UE) aims to detect hallucinated outputs of large language models (LLMs) to improve their reliability. However, UE metrics often exhibit unstable performance across configurations, which significan…

Uncertainty-Driven Loss for Single Image Super-Resolution

2021-12-01 · NeurIPS 2021 12 · Qian Ning, Weisheng Dong, Xin Li, Jinjian Wu 외

In low-level vision such as single image super-resolution (SISR), traditional MSE or L1 loss function treats every pixel equally with the assumption that the importance of all pixels is the same. However, it has been lon…

Image Super-Resolution Super-Resolution