MEMEN: Multi-layer Embedding with Memory Networks for Machine Comprehension
Machine comprehension(MC) style question answering is a representative problem in natural language processing. Previous methods rarely spend time on the improvement of encoding layer, especially the embedding of syntactic information and name entity of the words, which are very crucial to the quality of encoding. Moreover, existing attention methods represent each query word as a vector or use a single vector to represent the whole query sentence, neither of them can handle the proper weight of the key words in query sentence. In this paper, we introduce a novel neural network architecture called Multi-layer Embedding with Memory Network(MEMEN) for machine reading task. In the encoding layer, we employ classic skip-gram model to the syntactic and semantic information of the words to train a new kind of embedding layer. We also propose a memory network of full-orientation matching of the query and passage to catch more pivotal information. Experiments show that our model has competitive results both from the perspectives of precision and efficiency in Stanford Question Answering Dataset(SQuAD) among all published results and achieves the state-of-the-art results on TriviaQA dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Question AnsweringReading ComprehensionSentenceTriviaQAMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents
Recent GUI agents have made substantial progress in visual grounding and action prediction, yet they remain brittle in long-horizon tasks that require maintaining task state across many interface transitions. Existing ag…
Visual GroundingMemento: Reconstruct to Remember for Consistent Long Video Generation
Long-form video generation requires recurring subjects to remain consistent across various shots, viewpoints, motions, and scene transitions. Existing temporal decomposition methods improve scalability by generating vide…
Video GenerationMemEngine: A Unified and Modular Library for Developing Advanced Memory of LLM-based Agents
Recently, large language model based (LLM-based) agents have been widely applied across various fields. As a critical part, their memory capabilities have captured significant interest from both industrial and academic c…
Language ModelingLanguage ModellingLarge Language ModelMemento: Fine-tuning LLM Agents without Fine-tuning LLMs
In this paper, we introduce a novel learning paradigm for Adaptive Large Language Model (LLM) agents that eliminates the need for fine-tuning the underlying LLMs. Existing approaches are often either rigid, relying on st…
Reinforcement LearningMemento-Skills: Let Agents Design Agents
We introduce \emph{Memento-Skills}, a generalist, continually-learnable LLM agent system that functions as an \emph{agent-designing agent}: it autonomously constructs, adapts, and improves task-specific agents through ex…
Reinforcement LearningContinual Learning