Globally Normalized Transition-Based Neural Networks
We introduce a globally normalized transition-based neural network model that achieves state-of-the-art part-of-speech tagging, dependency parsing and sentence compression results. Our model is a simple feed-forward neural network that operates on a task-specific transition system, yet achieves comparable or better accuracies than recurrent models. We discuss the importance of global as opposed to local normalization: a key insight is that the label bias problem implies that globally normalized models can be strictly more expressive than locally normalized models.
Code (1)
Tasks
Dependency ParsingPart-Of-Speech TaggingSentenceSentence CompressionSimilar Papers 제목 키워드 기반
A Globally Normalized Neural Model for Semantic Parsing
In this paper, we propose a globally normalized model for context-free grammar (CFG)-based semantic parsing. Instead of predicting a probability, our model predicts a real-valued score at each step and does not suffer fr…
modelSemantic ParsingAn Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search
Globally normalized neural sequence models are considered superior to their locally normalized equivalents because they may ameliorate the effects of label bias. However, when considering high-capacity neural parametriza…
CCG SupertaggingMachine TranslationPhase transitions for the noisy transformer model in arbitrary dimension
We study the McKean--Vlasov free energy on the unit sphere associated with the unnormalized self-attention (USA) model for noisy transformer dynamics. We prove a sharp global-minimizer dichotomy in every dimension $d\ge2…
Dependency Parsing with Partial Annotations: An Empirical Comparison
This paper describes and compares two straightforward approaches for dependency parsing with partial annotations (PA). The first approach is based on a forest-based training objective for two CRF parsers, i.e., a biaffin…
Active LearningDependency ParsingGlobal Normalization for Streaming Speech Recognition in a Modular Framework
We introduce the Globally Normalized Autoregressive Transducer (GNAT) for addressing the label bias problem in streaming speech recognition. Our solution admits a tractable exact computation of the denominator for the se…
speech-recognitionSpeech Recognition