End-to-End Learning for Structured Prediction Energy Networks
Structured Prediction Energy Networks (SPENs) are a simple, yet expressive family of structured prediction models (Belanger and McCallum, 2016). An energy function over candidate structured outputs is given by a deep network, and predictions are formed by gradient-based optimization. This paper presents end-to-end learning for SPENs, where the energy function is discriminatively trained by back-propagating through gradient-based prediction. In our experience, the approach is substantially more accurate than the structured SVM method of Belanger and McCallum (2016), as it allows us to use more sophisticated non-convex energies. We provide a collection of techniques for improving the speed, accuracy, and memory requirements of end-to-end SPENs, and demonstrate the power of our method on 7-Scenes image denoising and CoNLL-2005 semantic role labeling tasks. In both, inexact minimization of non-convex SPEN energies is superior to baseline methods that use simplistic energy functions that can be minimized exactly.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingImage DenoisingPredictionSemantic Role LabelingStructured PredictionSimilar Papers 제목 키워드 기반
SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres
Event-centric structured prediction involves predicting structured outputs of events. In most NLP cases, event structures are complex with manifold dependency, and it is challenging to effectively represent these complic…
Event DetectionEvent Relation ExtractionPredictionStructured PredictionStructured Prediction Energy Networks
We introduce structured prediction energy networks (SPENs), a flexible framework for structured prediction. A deep architecture is used to define an energy function of candidate labels, and then predictions are produced …
General ClassificationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONPrediction+1Learning Discriminators as Energy Networks in Adversarial Learning
We propose a novel framework for structured prediction via adversarial learning. Existing adversarial learning methods involve two separate networks, i.e., the structured prediction models and the discriminative models, …
Image SegmentationMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONPrediction+2Structured Prediction in NLP -- A survey
Over the last several years, the field of Structured prediction in NLP has had seen huge advancements with sophisticated probabilistic graphical models, energy-based networks, and its combination with deep learning-based…
PredictionStructured PredictionSurveyText GenerationLearning Approximate Inference Networks for Structured Prediction
Structured prediction energy networks (SPENs; Belanger & McCallum 2016) use neural network architectures to define energy functions that can capture arbitrary dependencies among parts of structured outputs. Prior work us…
Language ModelingLanguage ModellingMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATION+3