Differentiable Greedy Networks
Optimal selection of a subset of items from a given set is a hard problem that requires combinatorial optimization. In this paper, we propose a subset selection algorithm that is trainable with gradient-based methods yet achieves near-optimal performance via submodular optimization. We focus on the task of identifying a relevant set of sentences for claim verification in the context of the FEVER task. Conventional methods for this task look at sentences on their individual merit and thus do not optimize the informativeness of sentences as a set. We show that our proposed method which builds on the idea of unfolding a greedy algorithm into a computational graph allows both interpretability and gradient-based training. The proposed differentiable greedy network (DGN) outperforms discrete optimization algorithms as well as other baseline methods in terms of precision and recall.
Code (0)
등록된 구현이 없습니다.
Tasks
Claim VerificationCombinatorial OptimizationInformativenessSimilar Papers 제목 키워드 기반
Differentiable Greedy Submodular Maximization: Guarantees, Gradient Estimators, and Applications
Motivated by, e.g., sensitivity analysis and end-to-end learning, the demand for differentiable optimization algorithms has been significantly increasing. In this paper, we establish a theoretically guaranteed versatile …
SensitivityDeep greedy unfolding: Sorting out argsorting in greedy sparse recovery algorithms
Gradient-based learning imposes (deep) neural networks to be differentiable at all steps. This includes model-based architectures constructed by unrolling iterations of an iterative algorithm onto layers of a neural netw…
Newton Greedy Pursuit: A Quadratic Approximation Method for Sparsity-Constrained Optimization
First-order greedy selection algorithms have been widely applied to sparsity-constrained optimization. The main theme of this type of methods is to evaluate the function gradient in the previous iteration to update the n…
parameter estimationNeural Machine Translation with Gumbel-Greedy Decoding
Previous neural machine translation models used some heuristic search algorithms (e.g., beam search) in order to avoid solving the maximum a posteriori problem over translation sentences at test time. In this paper, we p…
Heuristic SearchMachine TranslationTranslationDifferentiable Scheduled Sampling for Credit Assignment
We demonstrate that a continuous relaxation of the argmax operation can be used to create a differentiable approximation to greedy decoding for sequence-to-sequence (seq2seq) models. By incorporating this approximation i…
Machine Translationnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1