ConvGenVisMo: Evaluation of Conversational Generative Vision Models
Conversational generative vision models (CGVMs) like Visual ChatGPT (Wu et al., 2023) have recently emerged from the synthesis of computer vision and natural language processing techniques. These models enable more natural and interactive communication between humans and machines, because they can understand verbal inputs from users and generate responses in natural language along with visual outputs. To make informed decisions about the usage and deployment of these models, it is important to analyze their performance through a suitable evaluation framework on realistic datasets. In this paper, we present ConvGenVisMo, a framework for the novel task of evaluating CGVMs. ConvGenVisMo introduces a new benchmark evaluation dataset for this task, and also provides a suite of existing and new automated evaluation metrics to evaluate the outputs. All ConvGenVisMo assets, including the dataset and the evaluation code, will be made available publicly on GitHub.
Code (1)
Similar Papers 제목 키워드 기반
An Evaluation Protocol for Generative Conversational Systems
There is a multitude of novel generative models for open-domain conversational systems; however, there is no systematic evaluation of different systems. Systematic comparisons require consistency in experimental design, …
Experimental DesignConversational Bots for Psychotherapy: A Study of Generative Transformer Models Using Domain-specific Dialogues
Conversational bots have become non-traditional methods for therapy among individuals suffering from psychological illnesses. Leveraging deep neural generative language models, we propose a deep trainable neural conversa…
Response GenerationTransfer LearningFew-Shot Generative Conversational Query Rewriting
Conversational query rewriting aims to reformulate a concise conversational query to a fully specified, context-independent query that can be effectively handled by existing information retrieval systems. This paper pres…
Information RetrievalRetrievalSelf-Supervised LearningZero-Shot LearningFoundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI
Generative Artificial Intelligence is set to revolutionize healthcare delivery by transforming traditional patient care into a more personalized, efficient, and proactive process. Chatbots, serving as interactive convers…
EthicsMotionChain: Conversational Motion Controllers via Multimodal Prompts
Recent advancements in language models have demonstrated their adeptness in conducting multi-turn dialogues and retaining conversational context. However, this proficiency remains largely unexplored in other multimodal g…
Language ModelingLanguage ModellingMotion Generation