Sequence-to-sequence Models for Cache Transition Systems
In this paper, we present a sequence-to-sequence based approach for mapping natural language sentences to AMR semantic graphs. We transform the sequence to graph mapping problem to a word sequence to transition action sequence problem using a special transition system called a cache transition system. To address the sparsity issue of neural AMR parsing, we feed feature embeddings from the transition state to provide relevant local information for each decoder state. We present a monotonic hard attention model for the transition framework to handle the strictly left-to-right alignment between each transition state and the current buffer input focus. We evaluate our neural transition model on the AMR parsing task, and our parser outperforms other sequence-to-sequence approaches and achieves competitive results in comparison with the best-performing models.
Code (1)
Tasks
AMR ParsingDecoderHard AttentionSentence CompressionText SummarizationSimilar Papers 제목 키워드 기반
AMR-to-Text Generation with Cache Transition Systems
Text generation from AMR involves emitting sentences that reflect the meaning of their AMR annotations. Neural sequence-to-sequence models have successfully been used to decode strings from flattened graphs (e.g., using …
AMR-to-Text GenerationDecoderSentenceText GenerationA Transition-based Parser for Unscoped Episodic Logical Forms
"Episodic Logic:Unscoped Logical Form" (EL-ULF) is a semantic representation capturing predicate-argument structure as well as more challenging aspects of language within the Episodic Logic formalism. We present the firs…
Attamba: Attending To Multi-Token States
When predicting the next token in a sequence, vanilla transformers compute attention over all previous tokens, resulting in quadratic scaling of compute with sequence length. State-space models compress the entire sequen…
ChunkingState Space ModelsTreeKV: Smooth Key-Value Cache Compression with Tree Structures
Efficient key-value (KV) cache compression is critical for scaling transformer-based Large Language Models (LLMs) in long sequences and resource-limited settings. Existing methods evict tokens based on their positions or…
Language ModelingLanguage ModellingCacheback: Speculative Decoding With Nothing But Cache
We present Cacheback Decoding, a training-free and model-agnostic speculative decoding method that exploits the locality in language to accelerate Large Language Model (LLM) inference. Cacheback leverages only Least Rece…