Preference Based Adaptation for Learning Objectives

2018-12-01 · NeurIPS 2018 12 · Yao-Xiang Ding, Zhi-Hua Zhou

In many real-world learning tasks, it is hard to directly optimize the true performance measures, meanwhile choosing the right surrogate objectives is also difficult. Under this situation, it is desirable to incorporate an optimization of objective process into the learning loop based on weak modeling of the relationship between the true measure and the objective. In this work, we discuss the task of objective adaptation, in which the learner iteratively adapts the learning objective to the underlying true objective based on the preference feedback from an oracle. We show that when the objective can be linearly parameterized, this preference based learning problem can be solved by utilizing the dueling bandit model. A novel sampling based algorithm DL^2M is proposed to learn the optimal parameter, which enjoys strong theoretical guarantees and efficient empirical performance. To avoid learning a hypothesis from scratch after each objective function update, a boosting based hypothesis adaptation approach is proposed to efficiently adapt any pre-learned element hypothesis to the current objective. We apply the overall approach to multi-label learning, and show that the proposed approach achieves significant performance under various multi-label performance measures.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-Label Learning

Collaborative Personalized Preference Alignment for LLMs under Data Deficiency

2026-10-05 · Liyan Yang, Yige Yuan, Zhiqin Yang hf

Real-world users often exhibit highly heterogeneous preferences over multiple objectives for LLM responses. A lightweight aligner can tailor these responses to individual preferences, but scarce user-specific feedback ma…

An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift

2026-01-09 · Constantinos Karouzos, Xingwei Tan, Nikolaos Aletras arxiv

Preference tuning aligns base language models to human judgments of quality, helpfulness, or safety by optimizing over explicit preference signals rather than likelihood alone. Prior work has shown that preference tuning…

Dynamic Detection of Relevant Objectives and Adaptation to Preference Drifts in Interactive Evolutionary Multi-Objective Optimization

2024-11-07 · Seyed Mahdi Shavarani, Mahmoud Golabi, Richard Allmendinger, Lhassane Idoumghar

Evolutionary Multi-Objective Optimization Algorithms (EMOAs) are widely employed to tackle problems with multiple conflicting objectives. Recent research indicates that not all objectives are equally important to the dec…

Decision Making

An Offline Adaptation Framework for Constrained Multi-Objective Reinforcement Learning

2024-09-16 · Qian Lin, Zongkai Liu, Danying Mo, Chao Yu

In recent years, significant progress has been made in multi-objective reinforcement learning (RL) research, which aims to balance multiple objectives by incorporating preferences for each objective. In most existing stu…

Multi-Objective Reinforcement Learning Reinforcement Learning (RL)

A Generalized Algorithm for Multi-Objective Reinforcement Learning and Policy Adaptation

2019-08-21 · NeurIPS 2019 12 · Runzhe Yang, Xingyuan Sun, Karthik Narasimhan

We introduce a new algorithm for multi-objective reinforcement learning (MORL) with linear preferences, with the goal of enabling few-shot adaptation to new tasks. In MORL, the aim is to learn policies over multiple comp…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)