Sequential statistical inference for Large Language Models: Representation, validity, and monitoring
This discussion argues that sequential statistical inference can naturally contribute to LLM trustworthiness. In deployment, LLM systems are queried repeatedly, conditioned on evolving contexts, and incorporate user or tool feedback, and may exhibit behavioral shifts after model updates or distribution changes. The discussion is organized around three tasks: representation, modeling LLM interactions as dependent stochastic processes rather than isolated prompt--response pairs; validity, developing uncertainty guarantees that remain meaningful under dependence, repeated use, and adaptation; and monitoring, using sequential alarms and change-point detection to identify shifts in calibration, hallucination rates, refusal behavior, fairness, or other task-relevant properties. This perspective complements recent surveys by viewing trustworthy LLM deployment as a problem of statistical process control.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Asymptotic Representations for Sequential Decisions, Adaptive Experiments, and Batched Bandits
We develop asymptotic approximations that can be applied to sequential estimation and inference problems, adaptive randomized controlled trials, and related settings. In batched adaptive settings where the decision at on…
Adaptive Path-Integral Autoencoders: Representation Learning and Planning for Dynamical Systems
We present a representation learning algorithm that learns a low-dimensional latent dynamical system from high-dimensional sequential raw data, e.g., video. The framework builds upon recent advances in amortized inferenc…
Representation LearningVariational InferenceAdaptive Path-Integral Autoencoder: Representation Learning and Planning for Dynamical Systems
We present a representation learning algorithm that learns a low-dimensional latent dynamical system from high-dimensional \textit{sequential} raw data, e.g., video. The framework builds upon recent advances in amortized…
Representation LearningVariational InferenceAutoTool: Efficient Tool Selection for Large Language Model Agents
Large Language Model (LLM) agents have emerged as powerful tools for automating complex tasks by leveraging the reasoning and decision-making abilities of LLMs. However, a major bottleneck in current agent frameworks lie…
Online Tensor Inference
Recent technological advances have led to contemporary applications that demand real-time processing and analysis of sequentially arriving tensor data. Traditional offline learning, involving the storage and utilization …
Decision Making