Papers Language Modelling
“Language Modelling” 태그가 달린 논문 17,662편 · 필터 해제
Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models
Language models can be adapted by changing the computations applied to individual tokens. Quantum circuits offer one such approach, but evaluating wider circuits inside a large model can be computationally demanding. Her…
Language ModellingMoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-to-MLM Curriculum
Turkish encoder models have adopted modern architectures while leaving the pretraining objective fixed at masked language modelling. This paper introduces MoganBert-TR, a 149M-parameter Turkish encoder foundation model t…
Language ModellingFourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention
FFT-based spectral preprocessing of learned query-key (Q/K) projections substantially improves transformer attention on character-level language modelling. On TinyShakespeare: a fixed random spectral filter achieves val=…
Language ModellingAbstract representational geometry supports inference in large language models
A defining feature of human intelligence is the ability to adapt to changing environments by inferring latent task structure from sparse observations. Neuroscientific research indicates that this capability relies on the…
Language ModellingFine-Tuning Large Language Models for Quantum Reasoning
Large language models (LLMs) exhibit abilities beyond natural language modelling and text generation. Recent advances in their reasoning capabilities have spurred interest in applying LLMs to complex scientific tasks req…
Language ModellingText GenerationSpeech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation
Speech-to-text (S2T) systems for recognition (ASR) and translation (S2TT) typically generate discrete text tokens. In contrast, continuous-target language modelling performs generation in a continuous space, yet its pote…
Speech RecognitionLanguage ModellingRuntime-Certified Bounded-Error Quantized Attention
KV cache quantization reduces the memory cost of long-context LLM inference, but introduces approximation error that is typically validated only empirically. Existing systems rely on average-case robustness, with no mech…
Language ModellingScaling Categorical Flow Maps
Continuous diffusion and flow matching models could represent a powerful alternative to autoregressive approaches for language modelling (LM), as they unlock a host of advantages currently reserved for continuous modalit…
Language ModellingBiMol-Diff: A Unified Diffusion Framework for Molecular Generation and Captioning
Bridging molecular structures and natural language is essential for controllable design. Autoregressive models struggle with long-range dependencies, while standard diffusion processes apply uniform corruption across pos…
Molecule CaptioningLanguage ModellingMultilinguality at the Edge: Developing Language Models for the Global South
Where and how language models (LMs) are deployed determines who can benefit from them. However, there are several challenges that prevent effective deployment of LMs in non-English-speaking and hardware constrained commu…
Language ModellingLanguage, Place, and Social Media: Geographic Dialect Alignment in New Zealand
This thesis investigates geographic dialect alignment in place-informed social media communities, focussing on New Zealand-related Reddit communities. By integrating qualitative analyses of user perceptions with computat…
Language ModellingPredict and Reconstruct: Joint Objectives for Self-Supervised Language Representation Learning
Masked language modelling (MLM) has been the dominant pre-training objective for text encoders since BERT, yet it encourages representations that are strongly anchored to surface-form token identity rather than deeper se…
Representation LearningLanguage ModellingIs Biomedical Specialization Still Worth It? Insights from Domain-Adaptive Language Modelling with a New French Health Corpus
Large language models (LLMs) have demonstrated remarkable capabilities across diverse domains, yet their adaptation to specialized fields remains challenging, particularly for non-English languages. This study investigat…
Language ModellingDomain AdaptationLoRM: Learning the Language of Rotating Machinery for Self-Supervised Condition Monitoring
We present LoRM (Language of Rotating Machinery), a self-supervised framework for multi-modal rotating-machinery signal understanding and real-time condition monitoring. LoRM is built on the idea that rotating-machinery …
Language ModellingOn the Geometry of Positional Encodings in Transformers
Neural language models process sequences of words, but the mathematical operations inside them are insensitive to the order in which words appear. Positional encodings are the component added to remedy this. Despite thei…
Language Modelling'Layer su Layer': Identifying and Disambiguating the Italian NPN Construction in BERT's family
Interpretability research has highlighted the importance of evaluating Pretrained Language Models (PLMs) and in particular contextual embeddings against explicit linguistic theories to determine what linguistic informati…
Language ModellingA chemical language model for reticular materials design
Reticular chemistry has enabled the synthesis of tens of thousands of metal-organic frameworks (MOFs), yet the discovery of new materials still relies largely on intuition-driven linker design and iterative experimentati…
Language ModellingToward Generalized Cross-Lingual Hateful Language Detection with Web-Scale Data and Ensemble LLM Annotations
We study whether large-scale unlabelled web data and LLM-based synthetic annotations can improve multilingual hate speech detection. Starting from texts crawled via OpenWebSearch.eu~(OWS) in four languages (English, Germ…
Hate Speech DetectionLanguage ModellingRepetition Without Exclusivity: Scale Sensitivity of Referential Mechanisms in Child-Scale Language Models
We present the first systematic evaluation of mutual exclusivity (ME) -- the bias to map novel words to novel referents -- in text-only language models trained on child-directed speech. We operationalise ME as referentia…
Language ModellingSeparable neural architectures as a primitive for unified predictive and generative intelligence
Intelligent systems across physics, language and perception often exhibit factorisable structure, yet are typically modelled by monolithic neural architectures that do not explicitly exploit this structure. The separable…
Reinforcement LearningLanguage Modelling