Preference Communication in Multi-Objective Normal-Form Games
We consider preference communication in two-player multi-objective normal-form games. In such games, the payoffs resulting from joint actions are vector-valued. Taking a utility-based approach, we assume there exists a utility function for each player which maps vectors to scalar utilities and consider agents that aim to maximise the utility of expected payoff vectors. As agents typically do not know their opponent's utility function or strategy, they must learn policies to interact with each other. Inspired by Stackelberg games, we introduce four novel preference communication protocols to aid agents in arriving at adequate solutions. Each protocol describes a specific approach for one agent to communicate preferences over their actions and how another agent responds. Additionally, to study when communication emerges, we introduce a communication protocol where agents must learn when to communicate. These protocols are subsequently evaluated on a set of five benchmark games against baseline agents that do not communicate. We find that preference communication can alter the learning process and lead to the emergence of cyclic policies which had not been previously observed in this setting. We further observe that the resulting policies can heavily depend on the characteristics of the game that is played. Lastly, we find that communication naturally emerges in both cooperative and self-interested settings.
Code (1)
Tasks
FormSimilar Papers 제목 키워드 기반
Effects of Objective Normalization on Regions of Interest in Preference-Based Evolutionary Multi-Objective Optimization
Preference-based evolutionary multi-objective optimization (PBEMO) aims to approximate a region of interest (ROI) defined by the preference information from a decision maker (DM). Although objective functions in real-wor…
Investigating Normalization in Preference-based Evolutionary Multi-objective Optimization Using a Reference Point
Normalization of objectives plays a crucial role in evolutionary multi-objective optimization (EMO) to handle objective functions with different scales, which can be found in real-world problems. Although the effect of n…
Preference Orchestrator: Prompt-Aware Multi-Objective Alignment for Large Language Models
While Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks, aligning these models with varying human preferences across multiple objectives remains a sig…
Estimating Quality in Multi-Objective Bandits Optimization
Many real-world applications are characterized by a number of conflicting performance measures. As optimizing in a multi-objective setting leads to a set of non-dominated solutions, a preference function is required for …
Thompson SamplingGeneralized Multi-Objective Reinforcement Learning with Envelope Updates in URLLC-enabled Vehicular Networks
We develop a novel multi-objective reinforcement learning (MORL) framework to jointly optimize wireless network selection and autonomous driving policies in a multi-band vehicular network operating on conventional sub-6G…
Autonomous DrivingAutonomous VehiclesMulti-Objective Reinforcement Learning