Using multimodal resources for explanation approaches in intelligent systems
In this work we show that there is a need of using multimodal resources during human-computer interaction (HCI) in intelligent systems. We propose that not only creating multimodal output for the user is important, but to take multimodal input resources into account for the decision when and how to interact. Especially the use of multimodal input resources for the decision when and how to provide assistance in HCI is important. The use of assistive functionalities like providing adaptive explanations to keep the user motivated and cooperative is more than a side-effect and demands a closer look. In this paper we introduce our approach on how to use multimodal input ressources in an adaptive and generic explanation pipeline. We do not only concentrate on using explanations as a way to manage user knowledge, but to maintain the cooperativeness, trust and motivation of the user to continue a healthy and well-structured HCI.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Few-Shot Multimodal Explanation for Visual Question Answering
A key object in eXplainable Artificial Intelligence (XAI) is to create intelligent systems capable of reasoning and explaining real-world data to facilitate reliable decision-making. Recent studies have acknowledged the …
Explainable artificial intelligenceExplainable Artificial Intelligence (XAI)FS-MEVQAQuestion Answering+3Selective Survey: Most Efficient Models and Solvers for Integrative Multimodal Transport
In the family of Intelligent Transportation Systems (ITS), Multimodal Transport Systems (MMTS) have placed themselves as a mainstream transportation mean of our time as a feasible integrative transportation process. The …
A multimodal slice discovery framework for systematic failure detection and explanation in medical image classification
Despite advances in machine learning-based medical image classifiers, the safety and reliability of these systems remain major concerns in practical settings. Existing auditing approaches mainly rely on unimodal features…
Medical Image ClassificationExplanation GenerationEmotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier
The recent advancement of Multimodal Large Language Models (MLLMs) is transforming human-computer interaction (HCI) from surface-level exchanges into more nuanced and emotionally intelligent communication. To realize thi…
Multimodal Emotion RecognitionExplainable Human Activity Recognition: A Unified Review of Concepts and Mechanisms
Human activity recognition (HAR) has become a key component of intelligent systems for healthcare monitoring, assistive living, smart environments, and human-computer interaction. Although deep learning has substantially…
Human Activity Recognition