Language Modelling

Turkish encoder models have adopted modern architectures while leaving the pretraining objective fixed at masked language modelling. This paper introduces MoganBert-TR, a 149M-parameter Turkish encoder foundation model t…

Language Modelling

FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention

2026-07-08 · Athanasios Zeris arxiv

FFT-based spectral preprocessing of learned query-key (Q/K) projections substantially improves transformer attention on character-level language modelling. On TinyShakespeare: a fixed random spectral filter achieves val=…

Language Modelling

Abstract representational geometry supports inference in large language models

2026-06-22 · Yunan Zeng, Yuwang Wang arxiv

A defining feature of human intelligence is the ability to adapt to changing environments by inferring latent task structure from sparse observations. Neuroscientific research indicates that this capability relies on the…

Language Modelling

Fine-Tuning Large Language Models for Quantum Reasoning

2026-06-20 · Katherine Ip, Casey R. Myers, Udaya Parampalli, James Quach 외 arxiv

Large language models (LLMs) exhibit abilities beyond natural language modelling and text generation. Recent advances in their reasoning capabilities have spurred interest in applying LLMs to complex scientific tasks req…

Language Modelling Text Generation

Speech Meets ELF: Audio Conditional Continuous-Target Diffusion for Speech Recognition and Translation

2026-06-09 · Xuanchen Li, Tianrui Wang, Yuheng Lu, Zikang Huang 외 arxiv

Speech-to-text (S2T) systems for recognition (ASR) and translation (S2TT) typically generate discrete text tokens. In contrast, continuous-target language modelling performs generation in a continuous space, yet its pote…

Speech Recognition Language Modelling

Runtime-Certified Bounded-Error Quantized Attention

2026-05-20 · Dean Calver arxiv

KV cache quantization reduces the memory cost of long-context LLM inference, but introduces approximation error that is typically validated only empirically. Existing systems rely on average-case robustness, with no mech…

Language Modelling

전체 17,661편 보기 →