BSDAR: Beam Search Decoding with Attention Reward in Neural Keyphrase Generation
This study mainly investigates two decoding problems in neural keyphrase generation: sequence length bias and beam diversity. We introduce an extension of beam search inference based on word-level and n-gram level attention score to adjust and constrain Seq2Seq prediction at test time. Results show that our proposed solution can overcome the algorithm bias to shorter and nearly identical sequences, resulting in a significant improvement of the decoding performance on generating keyphrases that are present and absent in source text.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityKeyphrase GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models
Decoding methods for large language models often trade-off between diversity of outputs and parallelism of computation. Methods such as beam search and Gumbel top-k sampling can guarantee a different output for each elem…
DiversityLanguage ModelingLanguage ModellingLarge Language Model+1Integration of Frame- and Label-synchronous Beam Search for Streaming Encoder-decoder Speech Recognition
Although frame-based models, such as CTC and transducers, have an affinity for streaming automatic speech recognition, their decoding uses no future knowledge, which could lead to incorrect pruning. Conversely, label-bas…
Automatic Speech RecognitionDecoderspeech-recognitionSpeech RecognitionSegment-Level Vectorized Beam Search Based on Partially Autoregressive Inference
Attention-based encoder-decoder models with autoregressive (AR) decoding have proven to be the dominant approach for automatic speech recognition (ASR) due to their superior accuracy. However, they often suffer from slow…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Decoderspeech-recognition+1A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models
Beam search is a desirable choice of test-time decoding algorithm for neural sequence models because it potentially avoids search errors made by simpler greedy methods. However, typical cross entropy training procedures …
CCG SupertaggingMotion Segmentationnamed-entity-recognitionNamed Entity Recognition+1Tokyo Metropolitan University Neural Machine Translation System for WAT 2017
In this paper, we describe our neural machine translation (NMT) system, which is based on the attention-based NMT and uses long short-term memories (LSTM) as RNN. We implemented beam search and ensemble decoding in the N…
Machine TranslationNMTTranslation