Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing
Dynamic oracles provide strong supervision for training constituency parsers with exploration, but must be custom defined for a given parser's transition system. We explore using a policy gradient method as a parser-agnostic alternative. In addition to directly optimizing for a tree-level metric such as F1, policy gradient has the potential to reduce exposure bias by allowing exploration during training; moreover, it does not require a dynamic oracle for supervision. On four constituency parsers in three languages, the method substantially outperforms static oracle likelihood training in almost all settings. For parsers where a dynamic oracle is available (including a novel oracle which we define for the transition system of Dyer et al. 2016), policy gradient typically recaptures a substantial fraction of the performance gain afforded by the dynamic oracle.
Code (0)
등록된 구현이 없습니다.
Tasks
Constituency ParsingSimilar Papers 제목 키워드 기반
Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
Parsing accuracy using efficient greedy transition systems has improved dramatically in recent years thanks to neural networks. Despite striking results in dependency parsing, however, neural models have not surpassed st…
Constituency ParsingDependency ParsingRerankingSentenceBreaking the Computational Barrier: Provably Efficient Actor-Critic for Low-Rank MDPs
Reinforcement learning (RL) is a fundamental framework for sequential decision-making, in which an agent learns an optimal policy through interactions with an unknown environment. In settings with function approximation,…
Reinforcement LearningPolicy Improvement via Imitation of Multiple Oracles
Despite its promise, reinforcement learning's real-world adoption has been hampered by the need for costly exploration to learn a good policy. Imitation learning (IL) mitigates this shortcoming by using an oracle policy …
Imitation LearningMambaNeural Discontinuous Constituency Parsing
One of the most pressing issues in discontinuous constituency transition-based parsing is that the relevant information for parsing decisions could be located in any part of the stack or the buffer. In this paper, we pro…
Constituency ParsingBlending Imitation and Reinforcement Learning for Robust Policy Improvement
While reinforcement learning (RL) has shown promising performance, its sample complexity continues to be a substantial hurdle, restricting its broader application across a variety of domains. Imitation learning (IL) util…
Imitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)