Safety Bench: Identifying Safety-Sensitive Situations for Open-domain Conversational Systems
The social impact of natural language processing and its applications has received increasing attention. Here, we focus on the problem of safety for end-to-end conversational AI. We survey the problem landscape therein, introducing a taxonomy of three observed phenomena: the Instigator, Yea-Sayer, and Impostor effects. To help researchers better understand the impact of their conversational models with respect to these scenarios, we present Safety Bench, a set of open-source tooling for quickly assessing safety issues. Finally, we provide extensive analysis of these tools using five popular models and make recommendations for future use.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
AURA: Affordance-Understanding and Risk-aware Alignment Technique for Large Language Models
Present day LLMs face the challenge of managing affordance-based safety risks-situations where outputs inadvertently facilitate harmful actions due to overlooked logical implications. Traditional safety solutions, such a…
Autonomous Vehicles Meet the Physical World: RSS, Variability, Uncertainty, and Proving Safety (Expanded Version)
The Responsibility-Sensitive Safety (RSS) model offers provable safety for vehicle behaviors such as minimum safe following distance. However, handling worst-case variability and uncertainty may significantly lower vehic…
Autonomous VehiclesCollision AvoidanceSafeDriveRAG: Towards Safe Autonomous Driving with Knowledge Graph-based Retrieval-Augmented Generation
In this work, we study how vision-language models (VLMs) can be utilized to enhance the safety for the autonomous driving system, including perception, situational understanding, and path planning. However, existing rese…
Visual Question AnsweringInformation RetrievalAutonomous DrivingOn the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark
Dialogue safety problems severely limit the real-world deployment of neural conversational models and have attracted great research interests recently. However, dialogue safety problems remain under-defined and the corre…
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark
Dialogue safety problems severely limit the real-world deployment of neural conversational models and have attracted great research interests recently. However, dialogue safety problems remain under-defined and the corre…