Papers Text Infilling
“Text Infilling” 태그가 달린 논문 43편 · 필터 해제
The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMs
Diffusion-based large language models (dLLMs) have recently emerged as a powerful alternative to autoregressive LLMs, offering faster inference and greater interactivity via parallel decoding and bidirectional modeling. …
Code GenerationSafety AlignmentText GenerationText InfillingFlexible-length Text Infilling for Discrete Diffusion Models
Discrete diffusion models are a new class of text generators that offer advantages such as bidirectional context use, parallelizable generation, and flexible prompting compared to autoregressive models. However, a critic…
PositionText InfillingLaViDa: A Large Diffusion Language Model for Multimodal Understanding
Modern Vision-Language Models (VLMs) can solve a wide range of tasks requiring visual reasoning. In real-world scenarios, desirable properties for VLMs include fast inference and controllable generation (e.g., constraini…
Instruction FollowingLanguage ModelingLanguage ModellingText Infilling+1Insertion Language Models: Sequence Generation with Arbitrary-Position Insertions
Autoregressive models (ARMs), which predict subsequent tokens one-by-one ``from left to right,'' have achieved significant success across a wide range of sequence generation tasks. However, they struggle to accurately re…
DenoisingPositionText GenerationText InfillingEnhancing Spoken Discourse Modeling in Language Models Using Gestural Cues
Research in linguistics shows that non-verbal cues, such as gestures, play a crucial role in spoken discourse. For example, speakers perform hand gestures to indicate topic shifts, helping listeners identify transitions …
Language ModelingLanguage ModellingText InfillingTrajGPT: Controlled Synthetic Trajectory Generation Using a Multitask Transformer-Based Spatiotemporal Model
Human mobility modeling from GPS-trajectories and synthetic trajectory generation are crucial for various applications, such as urban planning, disaster management and epidemiology. Both of these tasks often require fill…
EpidemiologyText InfillingEmpowering Character-level Text Infilling by Eliminating Sub-Tokens
In infilling tasks, sub-tokens, representing instances where a complete token is segmented into two parts, often emerge at the boundaries of prefixes, middles, and suffixes. Traditional methods focused on training models…
Text InfillingTowards Probabilistically-Sound Beam Search with Masked Language Models
Beam search with masked language models (MLMs) is challenging in part because joint probability distributions over sequences are not readily available, unlike for autoregressive models. However, estimating such distribut…
Ancient Text RestorationText InfillingA Benchmark for Text Expansion: Datasets, Metrics, and Baselines
This work presents a new task of Text Expansion (TE), which aims to insert fine-grained modifiers into proper locations of the plain text to concretize or vivify human writings. Different from existing insertion-based wr…
2kInformativenessText InfillingA Simple yet Effective Framework for Few-Shot Aspect-Based Sentiment Analysis
The pre-training and fine-tuning paradigm has become the mainstream framework in the field of Aspect-Based Sentiment Analysis (ABSA). Although it has achieved sound performance in the domains containing enough fine-grain…
Aspect-Based Sentiment AnalysisAspect-Based Sentiment Analysis (ABSA)Aspect Term Extraction and Sentiment ClassificationSentiment Analysis+2Having Beer after Prayer? Measuring Cultural Bias in Large Language Models
As the reach of large language models (LMs) expands globally, their ability to cater to diverse cultural contexts becomes crucial. Despite advancements in multilingual capabilities, models are not designed with appropria…
named-entity-recognitionNamed Entity RecognitionNERSentiment Analysis+2Sequence-to-Sequence Pre-training with Unified Modality Masking for Visual Document Understanding
This paper presents GenDoc, a general sequence-to-sequence document understanding model pre-trained with unified masking across three modalities: text, image, and layout. The proposed model utilizes an encoder-decoder ar…
Decoderdocument understandingOptical Character Recognition (OCR)Text InfillingMAGVLT: Masked Generative Vision-and-Language Transformer
While generative modeling on multimodal image-text data has been actively developed with large-scale paired datasets, there have been limited attempts to generate both image and text data by a single model rather than a …
Image CaptioningImage GenerationImage to textPrediction+3Model-tuning Via Prompts Makes NLP Models Adversarially Robust
In recent years, NLP practitioners have converged on the following practice: (i) import an off-the-shelf pretrained (masked) language model; (ii) append a multilayer perceptron atop the CLS token's hidden representation …
Adversarial RobustnessLanguage ModellingText InfillingDon't Prompt, Search! Mining-based Zero-Shot Learning with Language Models
Masked language models like BERT can perform text classification in a zero-shot fashion by reformulating downstream tasks as text infilling. However, this approach is highly sensitive to the template used to prompt the m…
Text ClassificationText InfillingZero-Shot LearningZero-Shot Text ClassificationGenerative Prompt Tuning for Relation Classification
Using prompts to explore the knowledge contained within pre-trained language models for downstream tasks has now become an active topic. Current prompt tuning methods mostly convert the downstream tasks to masked languag…
ClassificationLanguage ModelingLanguage ModellingMasked Language Modeling+4MetaFill: Text Infilling for Meta-Path Generation on Heterogeneous Information Networks
Heterogeneous Information Network (HIN) is essential to study complicated networks containing multiple edge types and node types. Meta-path, a sequence of node types and edge types, is the core technique to embed HINs. S…
Graph EmbeddingLanguage ModellingLink PredictionNode Classification+1Reprogramming Pretrained Language Models for Antibody Sequence Infilling
Antibodies comprise the most versatile class of binding molecules, with numerous applications in biomedicine. Computational design of antibodies involves generating novel and diverse sequences, while maintaining structur…
DiversityLanguage ModellingSpecificityText InfillingA-TIP: Attribute-aware Text Infilling via Pre-trained Language Model
Text infilling aims to restore incomplete texts by filling in blanks, which has attracted more attention recently because of its wide application in ancient text restoration and text rewriting. However, attribute- aware …
Ancient Text RestorationAttributeLanguage ModelingLanguage Modelling+1Coordination Generation via Synchronized Text-Infilling
Generating synthetic data for supervised learning from large-scale pre-trained language models has enhanced performances across several NLP tasks, especially in low-resource scenarios. In particular, many studies of data…
Data AugmentationSentenceSentence ClassificationText Infilling