Should Robots be Obedient?
Intuitively, obedience -- following the order that a human gives -- seems like a good property for a robot to have. But, we humans are not perfect and we may give orders that are not best aligned to our preferences. We show that when a human is not perfectly rational then a robot that tries to infer and act according to the human's underlying preferences can always perform better than a robot that simply follows the human's literal order. Thus, there is a tradeoff between the obedience of a robot and the value it can attain for its owner. We investigate how this tradeoff is impacted by the way the robot infers the human's preferences, showing that some methods err more on the side of obedience than others. We then analyze how performance degrades when the robot has a misspecified model of the features that the human cares about or the level of rationality of the human. Finally, we study how robots can start detecting such model misspecification. Overall, our work suggests that there might be a middle ground in which robots intelligently decide when to obey human orders, but err on the side of obedience.
Code (1)
Similar Papers 제목 키워드 기반
On the Equilibrium Elicitation of Markov Games Through Information Design
This work considers a novel information design problem and studies how the craft of payoff-relevant environmental signals solely can influence the behaviors of intelligent agents. The agents' strategic interactions are c…
An Experimental Study on Learning Correlated Equilibrium in Routing Games
We study route choice in a repeated routing game where an uncertain state of nature determines link latency functions, and agents receive private route recommendation. The state is sampled in an i.i.d. manner in every ro…
DOST -- Domain Obedient Self-supervised Training for Multi Label Classification with Noisy Labels
The enormous demand for annotated data brought forth by deep learning techniques has been accompanied by the problem of annotation noise. Although this issue has been widely discussed in machine learning literature, it h…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONNevermind: Instruction Override and Moderation in Large Language Models
Given the impressive capabilities of recent Large Language Models (LLMs), we investigate and benchmark the most popular proprietary and different sized open source models on the task of explicit instruction following in …
Instruction FollowingLanguage ModelingLanguage ModellingShould Social Robots in Retail Manipulate Customers?
Against the backdrop of structural changes in the retail trade, social robots have found their way into retail stores and shopping malls in order to attract, welcome, and greet customers; to inform them, advise them, and…