Papers Language Modelling
“Language Modelling” 태그가 달린 논문 17,662편 · 필터 해제
Instruction set for the representation of graphs
We present IsalGraph, a method for representing the structure of any finite, simple graph as a compact string over a nine-character instruction alphabet. The encoding is executed by a small virtual machine comprising a s…
Language ModellingGraph SimilarityGraph GenerationLearning Long-Range Dependencies with Temporal Predictive Coding
Temporal Predictive Coding provides a layer-local, parallelisable mechanism for learning in recurrent systems, making it an attractive candidate for online local learning on neuromorphic and edge hardware. However, its r…
Language ModellingDialects of Translationese Shape Language Model Learning
Machine-translated data is widely used in multilingual NLP, particularly where native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it r…
Linguistic AcceptabilityLanguage ModellingTestimole-Conversational: A 30-Billion-Word Italian Discussion Board Corpus (1996-2024) for Language Modeling and Sociolinguistic Research
We present "Testimole-conversational" a massive collection of discussion boards messages in the Italian language. The large size of the corpus, more than 30B word-tokens (1996-2024), renders it an ideal dataset for nativ…
Language ModellingDomain AdaptationKalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking
State-space language models such as Mamba and gated linear attention (GLA) offer linear-complexity, parallelisable alternatives to transformers, but their linear state updates limit expressivity and robust state tracking…
Language ModellingContinual-learning for Modelling Low-Resource Languages from Large Language Models
Modelling a language model for a multi-lingual scenario includes several potential challenges, among which catastrophic forgetting is the major challenge. For example, small language models (SLM) built for low-resource l…
Visual Question AnsweringContinual LearningLanguage ModellingDemocratising Pathology Co-Pilots: An Open Pipeline and Dataset for Whole-Slide Vision-Language Modelling
Vision-language models (VLMs) have the potential to become co-pilots for pathologists. However, most VLMs either focus on small regions of interest within whole-slide images, provide only static slide-level outputs, or r…
Question AnsweringLanguage ModellingGeoreferencing complex relative locality descriptions with large language models
Georeferencing text documents has typically relied on either gazetteer-based methods to assign geographic coordinates to place names, or on language modelling approaches that associate textual terms with geographic locat…
Language ModellingGatedFWA: Linear Flash Windowed Attention with Gated Associative Memory
Modern autoregressive models rely on attention, yet the Softmax full attention in Transformers scales quadratically with sequence length. Sliding Window Attention (SWA) achieves linear-time encoding/decoding by constrain…
Language ModellingProvable Benefit of Sign Descent: A Minimal Model Under Heavy-Tailed Class Imbalance
Adaptive optimization methods (such as Adam) play a major role in LLM pretraining, significantly outperforming Gradient Descent (GD). Recent studies have proposed new smoothness assumptions on the loss function to explai…
Language ModellingAdam Simplified: Bias Correction Debunked
The Adam optimizer is a cornerstone of modern deep learning, yet the empirical necessity of each of its individual components is often taken for granted. This paper presents a focused investigation into the role of bias-…
Language ModellingEstranged Predictions: Measuring Semantic Category Disruption with Masked Language Modelling
This paper examines how science fiction destabilises ontological categories by measuring conceptual permeability across the terms human, animal, and machine using masked language modelling (MLM). Drawing on corpora of sc…
Language ModellingOn the Interplay between Positional Encodings, Morphological Complexity, and Word Order Flexibility
Language model architectures are predominantly first created for English and subsequently applied to other languages. It is an open question whether this architectural bias leads to degraded performance for languages tha…
Language ModellingExpert Merging in Sparse Mixture of Experts with Nash Bargaining
Existing expert merging strategies for Sparse Mixture of Experts (SMoE) typically rely on input-dependent or input-independent averaging of expert parameters, but often lack a principled weighting mechanism. In this work…
Image ClassificationText ClassificationLanguage ModellingLayout-Independent License Plate Recognition via Integrated Vision and Language Models
This work presents a pattern-aware framework for automatic license plate recognition (ALPR), designed to operate reliably across diverse plate layouts and challenging real-world conditions. The proposed system consists o…
License Plate RecognitionLanguage ModellingParsVoice: A Large-Scale Multi-Speaker Persian Speech Corpus for Text-to-Speech Synthesis
Persian remains substantially underrepresented in open speech-text resources, limiting progress in multi-speaker text-to-speech (TTS), speech-language modelling, and low-resource speech processing. We introduce ParsVoice…
Text-To-Speech SynthesisSpeaker IdentificationLanguage ModellingLooking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
Training vision-language models on cognitively-plausible amounts of data requires rethinking how models integrate multimodal information. Within the constraints of the Vision track for the BabyLM Challenge 2025, we propo…
Language ModellingVisual GroundingCOLE: a Comprehensive Benchmark for French Language Understanding Evaluation
To address the need for a more comprehensive evaluation of French Natural Language Understanding (NLU), we introduce COLE, a new benchmark composed of 23 diverse task covering a broad range of NLU capabilities, including…
Natural Language UnderstandingWord Sense DisambiguationSentiment AnalysisLanguage ModellingSigma: Semantically Informative Pre-training for Skeleton-based Sign Language Understanding
Pre-training has proven effective for learning transferable features in sign language understanding (SLU) tasks. Recently, skeleton-based methods have gained increasing attention because they can robustly handle variatio…
Sign Language TranslationSign Language RecognitionContrastive LearningLanguage ModellingCharting a Decade of Computational Linguistics in Italy: The CLiC-it Corpus
Over the past decade, Computational Linguistics (CL) and Natural Language Processing (NLP) have evolved rapidly, especially with the advent of Transformer-based Large Language Models (LLMs). This shift has transformed re…
Language Modelling