States of LLM-generated Texts and Phase Transitions between them
It is known for some time that autocorrelations of words in human-written texts decay according to a power law. Recent works have also shown that the autocorrelations decay in texts generated by LLMs is qualitatively different from the literary texts. Solid state physics tie the autocorrelations decay laws to the states of matter. In this work, we empirically demonstrate that, depending on the temperature parameter, LLMs can generate text that can be classified as solid, critical state or gas.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Critical Phase Transition in Large Language Models
Large Language Models (LLMs) have demonstrated impressive performance. To understand their behaviors, we need to consider the fact that LLMs sometimes show qualitative changes. The natural world also presents such change…
Language ModellingLarge Language ModelAttention to Order: Transformers Discover Phase Transitions via Learnability
Phase transitions mark qualitative reorganizations of collective behavior, yet identifying their boundaries remains challenging whenever analytic solutions are absent and conventional simulations fail. Here we introduce …
Self-Supervised LearningAdiabatic Fine-Tuning of Neural Quantum States Enables Detection of Phase Transitions in Weight Space
Neural quantum states (NQS) have emerged as a powerful tool for approximating quantum wavefunctions using deep learning. While these models achieve remarkable accuracy, understanding how they encode physical information …
Learning to Ball: Composing Policies for Long-Horizon Basketball Moves
Learning a control policy for a multi-phase, long-horizon task, such as basketball maneuvers, remains challenging for reinforcement learning approaches due to the need for seamless policy composition and transitions betw…
Reinforcement LearningTopological Persistence Machine of Phase Transitions
The study of phase transitions using data-driven approaches is challenging, especially when little prior knowledge of the system is available. Topological data analysis is an emerging framework for characterizing the sha…
Topological Data Analysis