The Contribution of XAI for the Safe Development and Certification of AI: An Expert-Based Analysis
Developing and certifying safe - or so-called trustworthy - AI has become an increasingly salient issue, especially in light of upcoming regulation such as the EU AI Act. In this context, the black-box nature of machine learning models limits the use of conventional avenues of approach towards certifying complex technical systems. As a potential solution, methods to give insights into this black-box - devised in the field of eXplainable AI (XAI) - could be used. In this study, the potential and shortcomings of such methods for the purpose of safe AI development and certification are discussed in 15 qualitative interviews with experts out of the areas of (X)AI and certification. We find that XAI methods can be a helpful asset for safe AI development, as they can show biases and failures of ML-models, but since certification relies on comprehensive and correct information about technical systems, their impact is expected to be limited.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development
Despite large progress in Explainable and Safe AI, practitioners suffer from a lack of regulation and standards for AI safety. In this work we merge recent regulation efforts by the European Union and first proposals for…
Document AIDynamic directed functional connectivity as a neural biomarker for objective motor skill assessment
Objective motor skill assessment plays a critical role in fields such as surgery, where proficiency is vital for certification and patient safety. Existing assessment methods, however, rely heavily on subjective human ju…
EEGFunctional ConnectivitySpecificityCPS Engineering: Gap Analysis and Perspectives
Virtualization of computing and networking, IT-OT convergence, cybersecurity and AI-based enhancement of autonomy are significantly increasing the complexity of CPS and CPSoS. New challenges have emerged to demonstrate t…
RISC-V Functional Safety for Autonomous Automotive Systems: An Analytical Framework and Research Roadmap for ML-Assisted Certification
RISC-V is emerging as a viable platform for automotive-grade embedded computing, with recent ISO 26262 ASIL-D certifications demonstrating readiness for safety-critical deployment in autonomous driving systems. However, …
Reinforcement LearningAutonomous DrivingCertification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study
Auditing plays a pivotal role in the development of trustworthy AI. However, current research primarily focuses on creating auditable AI documentation, which is intended for regulators and experts rather than end-users a…
Survey