Papers Language Modelling
“Language Modelling” 태그가 달린 논문 17,662편 · 필터 해제
Developing an AI framework to automatically detect shared decision-making in patient-doctor conversations
Shared decision-making (SDM) is necessary to achieve patient-centred care. Currently no methodology exists to automatically measure SDM at scale. This study aimed to develop an automated approach to measure SDM by using …
Language ModellingReversible Deep Equilibrium Models
Deep Equilibrium Models (DEQs) are an interesting class of implicit model where the model output is implicitly defined as the fixed point of a learned function. These models have been shown to outperform explicit (fixed-…
Image ClassificationLanguage ModellingSemantic Fusion with Fuzzy-Membership Features for Controllable Language Modelling
We propose semantic fusion, a lightweight scheme that augments a Transformer language model (LM) with a parallel, fuzzy-membership feature channel that encodes token-level semantics. Each token is represented by a vector…
Language ModellingInvestigating Transcription Normalization in the Faetar ASR Benchmark
We examine the role of transcription inconsistencies in the Faetar Automatic Speech Recognition benchmark, a challenging low-resource ASR benchmark. With the help of a small, hand-constructed lexicon, we conclude that fi…
Speech RecognitionLanguage ModellingALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Models
Large Language Models (LLMs) have shown remarkable performance across a wide range of natural language processing tasks. Quality Estimation (QE) for Machine Translation (MT), which assesses the quality of a source-target…
Machine TranslationLanguage ModellingFractal Language Modelling by Universal Sequence Maps (USM)
Motivation: With the advent of Language Models using Transformers, popularized by ChatGPT, there is a renewed interest in exploring encoding procedures that numerically represent symbolic sequences at multiple scales and…
Language ModellingH-Net++: Hierarchical Dynamic Chunking for Tokenizer-Free Language Modelling in Morphologically-Rich Languages
Byte-level language models eliminate fragile tokenizers but face computational challenges in morphologically-rich languages (MRLs), where words span many bytes. We propose H-NET++, a hierarchical dynamic-chunking model t…
Computational EfficiencyLanguage ModellingHuman-like fleeting memory improves language learning but impairs reading time prediction in transformer language models
Human memory is fleeting. As words are processed, the exact wordforms that make up incoming sentences are rapidly lost. Cognitive scientists have long believed that this limitation of memory may, paradoxically, help in l…
Language ModellingRooseBERT: A New Deal For Political Language Modelling
The increasing amount of political debates and politics-related discussions calls for the definition of novel computational methods to automatically analyse such content with the final goal of lightening up political del…
Sentiment AnalysisLanguage ModellingStance DetectionUPLME: Uncertainty-Aware Probabilistic Language Modelling for Robust Empathy Regression
Noisy self-reported empathy scores challenge supervised learning for empathy regression. While many algorithms have been proposed for learning with noisy labels in textual classification problems, the regression counterp…
Learning with noisy labelsLanguage ModellingSmilesT5: Domain-specific pretraining for molecular language models
Molecular property prediction is an increasingly critical task within drug discovery and development. Typically, neural networks can learn molecular properties using graph-based, language-based or feature-based methods. …
Molecular Property PredictionComputational EfficiencyLanguage ModellingDrug DiscoveryA Deep Learning Automatic Speech Recognition Model for Shona Language
This study presented the development of a deep learning-based Automatic Speech Recognition system for Shona, a low-resource language characterized by unique tonal and grammatical complexities. The research aimed to addre…
Speech RecognitionLanguage ModellingTransfer LearningData AugmentationVisual-Language Model Knowledge Distillation Method for Image Quality Assessment
Image Quality Assessment (IQA) is a core task in computer vision. Multimodal methods based on vision-language models, such as CLIP, have demonstrated exceptional generalization capabilities in IQA tasks. To address the i…
Image Quality AssessmentKnowledge DistillationLanguage ModelingLanguage ModellingMaking Language Model a Hierarchical Classifier and Generator
Decoder-only language models, such as GPT and LLaMA, generally decode on the last layer. Motivated by human's hierarchical thinking capability, we propose that a hierarchical decoder architecture could be built with diff…
DecoderLanguage ModelingLanguage Modellingmodel+3VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning
Recent advancements in vision-language models (VLMs) have improved performance by increasing the number of visual tokens, which are often significantly longer than text tokens. However, we observe that most real-world sc…
Language ModelingLanguage ModellingOptical Character Recognition (OCR)reinforcement-learning+2The Generative Energy Arena (GEA): Incorporating Energy Awareness in Large Language Model (LLM) Human Evaluations
The evaluation of large language models is a complex task, in which several approaches have been proposed. The most common is the use of automated benchmarks in which LLMs have to answer multiple-choice questions of diff…
Language ModelingLanguage ModellingLarge Language ModelMultiple-choiceInverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities
In the era of Large Language Models (LLMs), alignment has emerged as a fundamental yet challenging problem in the pursuit of more reliable, controllable, and capable machine intelligence. The recent success of reasoning …
Language ModelingLanguage ModellingLarge Language ModelReinforcement Learning (RL)Assay2Mol: large language model-based drug design using BioAssay context
Scientific databases aggregate vast amounts of quantitative data alongside descriptive text. In biochemistry, molecule screening assays evaluate the functional responses of candidate molecules against disease targets. Un…
DescriptiveDrug DesignDrug DiscoveryIn-Context Learning+3Describe Anything Model for Visual Question Answering on Text-rich Images
Recent progress has been made in region-aware vision-language modeling, particularly with the emergence of the Describe Anything Model (DAM). DAM is capable of generating detailed descriptions of any specific image areas…
DescriptiveLanguage ModelingLanguage ModellingQuestion Answering+2InstructFLIP: Exploring Unified Vision-Language Model for Face Anti-spoofing
Face anti-spoofing (FAS) aims to construct a robust system that can withstand diverse attacks. While recent efforts have concentrated mainly on cross-domain generalization, two significant challenges persist: limited sem…
Domain GeneralizationFace Anti-SpoofingLanguage ModelingLanguage Modelling