Adversarial Generation and Encoding of Nested Texts
In this paper we propose a new language model called AGENT, which stands for Adversarial Generation and Encoding of Nested Texts. AGENT is designed for encoding, generating and refining documents that consist of a long and coherent text, such as an entire book, provided they are hierarchically annotated (nested). i.e. divided into sentences, paragraphs and chapters. The core idea of our system is learning vector representations for each level of the text hierarchy (sentences, paragraphs, etc...), and train each such representation to perform 3 tasks: The task of reconstructing the sequence of vectors from a lower level that was used to create the representation, and generalized versions of the Masked Language Modeling (MLM) and "Next Sentence Prediction" tasks from BERT Devlin et al. [2018]. Additionally we present a new adversarial model for long text generation and suggest a way to improve the coherence of the generated text by traversing its vector representation tree.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingMasked Language ModelingSentenceText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Nested Variational Autoencoder for Topic Modeling on Microtexts with Word Vectors
Most of the information on the Internet is represented in the form of microtexts, which are short text snippets such as news headlines or tweets. These sources of information are abundant, and mining these data could unc…
Topic ModelsWord EmbeddingsTSCheater: Generating High-Quality Tibetan Adversarial Texts via Visual Similarity
Language models based on deep neural networks are vulnerable to textual adversarial attacks. While rich-resource languages like English are receiving focused attention, Tibetan, a cross-border language, is gradually bein…
Adversarial RobustnessAdversarial TextSemantic SimilaritySemantic Textual Similarity+1Entity-to-Text based Data Augmentation for various Named Entity Recognition Tasks
Data augmentation techniques have been used to alleviate the problem of scarce labeled data in various NER tasks (flat, nested, and discontinuous NER tasks). Existing augmentation techniques either manipulate the words i…
Data AugmentationDiversitynamed-entity-recognitionNamed Entity Recognition+3APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding
Context-augmented generation (CAG) techniques, including RAG and ICL, require the efficient combination of multiple contexts to generate responses to user queries. Directly inputting these contexts as a sequence introduc…
RAGHuman-in-the-Loop Generation of Adversarial Texts: A Case Study on Tibetan Script
DNN-based language models perform excellently on various tasks, but even SOTA LLMs are susceptible to textual adversarial attacks. Adversarial texts play crucial roles in multiple subfields of NLP. However, current resea…
Adversarial AttackAdversarial Robustness