Semi-Autoregressive Transformer for Image Captioning
Current state-of-the-art image captioning models adopt autoregressive decoders, \ie they generate each word by conditioning on previously generated words, which leads to heavy latency during inference. To tackle this issue, non-autoregressive image captioning models have recently been proposed to significantly accelerate the speed of inference by generating all words in parallel. However, these non-autoregressive models inevitably suffer from large generation quality degradation since they remove words dependence excessively. To make a better trade-off between speed and quality, we introduce a semi-autoregressive model for image captioning~(dubbed as SATIC), which keeps the autoregressive property in global but generates words parallelly in local . Based on Transformer, there are only a few modifications needed to implement SATIC. Experimental results on the MSCOCO image captioning benchmark show that SATIC can achieve a good trade-off without bells and whistles. Code is available at {\color{magenta}\url{https://github.com/YuanEZhou/satic}}.
Code (1)
Tasks
Image CaptioningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Semi-Autoregressive Image Captioning
Current state-of-the-art approaches for image captioning typically adopt an autoregressive manner, i.e., generating descriptions word by word, which suffers from slow decoding issue and becomes a bottleneck in real-time …
DecoderImage CaptioningSentenceBounding and Filling: A Fast and Flexible Framework for Image Captioning
Most image captioning models following an autoregressive manner suffer from significant inference latency. Several models adopted a non-autoregressive manner to speed up the process. However, the vanilla non-autoregressi…
Image CaptioningImage DescriptionNon-Autoregressive Image Captioning with Counterfactuals-Critical Multi-Agent Learning
Most image captioning models are autoregressive, i.e. they generate each word by conditioning on previously generated words, which leads to heavy latency during inference. Recently, non-autoregressive decoding has been p…
Image CaptioningMachine TranslationMulti-agent Reinforcement LearningSentence+1Semantic-Conditional Diffusion Networks for Image Captioning
Recent advances on text-to-image generation have witnessed the rise of diffusion models which act as powerful generative models. Nevertheless, it is not trivial to exploit such latent variable models to capture the depen…
Cross-Modal RetrievalDecoderImage CaptioningImage Generation+4Length-Controllable Image Captioning
The last decade has witnessed remarkable progress in the image captioning task; however, most existing methods cannot control their captions, \emph{e.g.}, choosing to describe the image either roughly or in detail. In th…
controllable image captioningDecoderDiversityImage Captioning