Structured Prediction as Translation between Augmented Natural Languages
We propose a new framework, Translation between Augmented Natural Languages (TANL), to solve many structured prediction language tasks including joint entity and relation extraction, nested named entity recognition, relation classification, semantic role labeling, event extraction, coreference resolution, and dialogue state tracking. Instead of tackling the problem by training task-specific discriminative classifiers, we frame it as a translation task between augmented natural languages, from which the task-relevant information can be easily extracted. Our approach can match or outperform task-specific models on all tasks, and in particular, achieves new state-of-the-art results on joint entity and relation extraction (CoNLL04, ADE, NYT, and ACE2005 datasets), relation classification (FewRel and TACRED), and semantic role labeling (CoNLL-2005 and CoNLL-2012). We accomplish this while using the same architecture and hyperparameters for all tasks and even when training a single model to solve all tasks at the same time (multi-task learning). Finally, we show that our framework can also significantly improve the performance in a low-resource regime, thanks to better use of label semantics.
Code (2)
Tasks
coreference-resolutionCoreference ResolutionDialogue State TrackingEvent ExtractionJoint Entity and Relation ExtractionMulti-Task Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)Nested Named Entity RecognitionPredictionRelationRelation ClassificationRelation ExtractionSemantic Role LabelingStructured PredictionTranslationSimilar Papers 제목 키워드 기반
Reward Augmented Maximum Likelihood for Neural Structured Prediction
A key problem in structured output prediction is direct optimization of the task reward function that matters for test evaluation. This paper presents a simple and computationally efficient approach to incorporate task r…
Machine TranslationPredictionspeech-recognitionSpeech Recognition+2DeepSPIN: Deep Structured Prediction for Natural Language Processing
DeepSPIN is a research project funded by the European Research Council (ERC) whose goal is to develop new neural structured prediction methods, models, and algorithms for improving the quality, interpretability, and data…
Machine TranslationPredictionStructured PredictionTranslationReasonSTL: Bridging Natural Language and Signal Temporal Logic via Tool-Augmented Process-Rewarded Learning
Signal Temporal Logic (STL) is an expressive formal language for specifying spatio-temporal requirements over real-valued, real-time signals. It has been widely used for the verification and synthesis of autonomous syste…
Beyond MLE: Investigating SEARNN for Low-Resourced Neural Machine Translation
Structured prediction tasks, like machine translation, involve learning functions that map structured inputs to structured outputs. Recurrent Neural Networks (RNNs) have historically been a popular choice for such tasks,…
Machine TranslationStructured PredictionTranslationRetrieval-Augmented Machine Translation with Unstructured Knowledge
Retrieval-augmented generation (RAG) introduces additional information to enhance large language models (LLMs). In machine translation (MT), previous work typically retrieves in-context examples from paired MT corpora, o…
Knowledge GraphsMachine TranslationRAGRetrieval+3