Spontaneous Emerging Preference in Two-tower Language Model
The ever-growing size of the foundation language model has brought significant performance gains in various types of downstream tasks. With the existence of side-effects brought about by the large size of the foundation language model such as deployment cost, availability issues, and environmental cost, there is some interest in exploring other possible directions, such as a divide-and-conquer scheme. In this paper, we are asking a basic question: are language processes naturally dividable? We study this problem with a simple two-tower language model setting, where two language models with identical configurations are trained side-by-side cooperatively. With this setting, we discover the spontaneous emerging preference phenomenon, where some of the tokens are consistently better predicted by one tower while others by another tower. This phenomenon is qualitatively stable, regardless of model configuration and type, suggesting this as an intrinsic property of natural language. This study suggests that interesting properties of natural language are still waiting to be discovered, which may aid the future development of natural language processing techniques.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingVocal Bursts Valence PredictionSimilar Papers 제목 키워드 기반
SLIDE: Integrating Speech Language Model with LLM for Spontaneous Spoken Dialogue Generation
Recently, ``textless" speech language models (SLMs) based on speech units have made huge progress in generating naturalistic speech, including non-verbal vocalizations. However, the generated speech samples often lack se…
Dialogue GenerationLanguage ModelingLanguage ModellingTowerMind: A Tower Defence Game Learning Environment and Benchmark for LLM as Agents
Recent breakthroughs in Large Language Models (LLMs) have positioned them as a promising paradigm for agents, with long-term planning and decision-making emerging as core general-purpose capabilities for adapting to dive…
Reinforcement LearningAccuracy meets Diversity in a News Recommender System
News recommender systems face certain challenges. These challenges arise due to evolving users’ preferences over dynamically created news articles. The diversity is necessary for a news recommender system to expose users…
ArticlesDiversityRecommendation SystemsZero Shot on the Cold-Start Problem: Model-Agnostic Interest Learning for Recommender Systems
User behavior has been validated to be effective in revealing personalized preferences for commercial recommendations. However, few user-item interactions can be collected for new users, which results in a null space for…
Recommendation SystemsSpontaneous Reward Hacking in Iterative Self-Refinement
Language models are capable of iteratively improving their outputs based on natural language feedback, thus enabling in-context optimization of user preference. In place of human users, a second language model can be use…
Language ModelingLanguage Modelling