paper-with-me

홈 › Papers

Stability as a Liability:Systematic Breakdown of Linguistic Structure in LLMs

2026-01-26 · Xianzhe Meng, Qiangsheng Zeng, Ling Luo, Qinghan Yang, Jiarui Hao, Wenbo Wu, Qinyu Wang, Rui Yin, Lin Qi, Renzhi Lu arxiv

Training stability is typically regarded as a prerequisite for reliable optimization in large language models. In this work, we analyze how stabilizing training dynamics affects the induced generation distribution. We show that under standard maximum likelihood training, stable parameter trajectories lead stationary solutions to approximately minimize the forward KL divergence to the empirical distribution, while implicitly reducing generative entropy. As a consequence, the learned model can concentrate probability mass on a limited subset of empirical modes, exhibiting systematic degeneration despite smooth loss convergence. We empirically validate this effect using a controlled feedback-based training framework that stabilizes internal generation statistics, observing consistent low-entropy outputs and repetitive behavior across architectures and random seeds. It indicates that optimization stability and generative expressivity are not inherently aligned, and that stability alone is an insufficient indicator of generative quality.

📄 PDF Abstract BibTeX arXiv:2601.18588

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

A Case for Lifetime Reliability-Aware Neuromorphic Computing

2020-07-04 · Shihao Song, Anup Das

Neuromorphic computing with non-volatile memory (NVM) can significantly improve performance and lower energy consumption of machine learning tasks implemented using spike-based computations and bio-inspired learning algo…

BIG-bench Machine Learning

Measurement-Driven Early Warning of Reliability Breakdown in 5G NSA Railway Networks

2025-11-12 · Po-Heng Chou, Da-Chih Lin, Hung-Yu Wei, Walid Saad 외 arxiv

This paper presents a measurement-driven study of early warning for reliability breakdown events in 5G non-standalone (NSA) railway networks. Using 10~Hz metro-train measurement traces with serving- and neighbor-cell ind…

Embodiment-Induced Coordination Regimes in Tabular Multi-Agent Q-Learning

2026-01-24 · Muhammad Ahmed Atif, Nehal Naeem Haji, Mohammad Shahid Shaikh, Muhammad Ebad Atif arxiv

Centralized value learning is often assumed to improve coordination and stability in multi-agent reinforcement learning, yet this assumption is rarely tested under controlled conditions. We directly evaluate it in a full…

Multi-agent Reinforcement LearningRepresentation Learning

AI Healthcare Chatbots as Information Infrastructure: A Large-Scale Study of User-Reported Breakdowns

2026-06-25 · Muhammad Hassan, Ramazan Yener, Ece Gumusel, Masooda Bashir arxiv

AI healthcare chatbots are increasingly used to support health information seeking and self-management, yet their performance and impact on users remains to be studied. This study examines over 15,000 user reviews from 5…

Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability

2026-05-11 · Harsh Raj, Niranjan Orkat, Suvrorup Mukherjee, Aritra Guha 외 arxiv

This paper establishes a rigorous measurement science for AI agent reliability, providing a foundational framework for quantifying consistency under semantically preserving perturbations. By leveraging $U$-statistics for…