Are Current Decoding Strategies Capable of Facing the Challenges of Visual Dialogue?
Decoding strategies play a crucial role in natural language generation systems. They are usually designed and evaluated in open-ended text-only tasks, and it is not clear how different strategies handle the numerous challenges that goal-oriented multimodal systems face (such as grounding and informativeness). To answer this question, we compare a wide variety of different decoding strategies and hyper-parameter configurations in a Visual Dialogue referential game. Although none of them successfully balance lexical richness, accuracy in the task, and visual grounding, our in-depth analysis allows us to highlight the strengths and weaknesses of each decoding strategy. We believe our findings and suggestions may serve as a starting point for designing more effective decoding algorithms that handle the challenges of Visual Dialogue tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
InformativenessText GenerationVisual GroundingSimilar Papers 제목 키워드 기반
LLM can Achieve Self-Regulation via Hyperparameter Aware Generation
In the realm of Large Language Models (LLMs), users commonly employ diverse decoding strategies and adjust hyperparameters to control the generated text. However, a critical question emerges: Are LLMs conscious of the ex…
Text GenerationResearch on False Data Injection Attacks in VSC-HVDC Systems
The false data injection (FDI) attack is a crucial form of cyber-physical security problems facing cyber-physical power systems. However, there is no research revealing the problem of FDI attacks facing voltage source co…
FormWirelessAgent: Large Language Model Agents for Intelligent Wireless Networks
Wireless networks are increasingly facing challenges due to their expanding scale and complexity. These challenges underscore the need for advanced AI-driven strategies, particularly in the upcoming 6G networks. In this …
Decision MakingLanguage ModelingLanguage ModellingLarge Language Model+1Surfacing Biases in Large Language Models using Contrastive Input Decoding
Ensuring that large language models (LMs) are fair, robust and useful requires an understanding of how different modifications to their inputs impact the model's behaviour. In the context of open-text generation tasks, h…
Text GenerationDecoding surface codes with deep reinforcement learning and probabilistic policy reuse
Quantum computing (QC) promises significant advantages on certain hard computational tasks over classical computers. However, current quantum hardware, also known as noisy intermediate-scale quantum computers (NISQ), are…
Deep Reinforcement LearningQ-Learningreinforcement-learningReinforcement Learning (RL)