Distributed Multi-Task Learning for Stochastic Bandits with Context Distribution and Stage-wise Constraints

2024-01-21 · Jiabin Lin, Shana Moothedath

We present conservative distributed multi-task learning in stochastic linear contextual bandits with heterogeneous agents. This extends conservative linear bandits to a distributed setting where M agents tackle different but related tasks while adhering to stage-wise performance constraints. The exact context is unknown, and only a context distribution is available to the agents as in many practical applications that involve a prediction mechanism to infer context, such as stock market prediction and weather forecast. We propose a distributed upper confidence bound (UCB) algorithm, DiSC-UCB. Our algorithm constructs a pruned action set during each round to ensure the constraints are met. Additionally, it includes synchronized sharing of estimates among agents via a central server using well-structured synchronization steps. We prove the regret and communication bounds on the algorithm. We extend the problem to a setting where the agents are unaware of the baseline reward. For this setting, we provide a modified algorithm, DiSC-UCB2, and we show that the modified algorithm achieves the same regret and communication bounds. We empirically validated the performance of our algorithm on synthetic data and real-world Movielens-100K data.

📄 PDF Abstract BibTeX arXiv:2401.11563

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-Armed Bandits Multi-Task Learning Stock Market Prediction

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Federated Learning for Heterogeneous Bandits with Unobserved Contexts

2023-03-29 · Jiabin Lin, Shana Moothedath

We study the problem of federated stochastic multi-arm contextual bandits with unknown contexts, in which M agents are faced with different bandits and collaborate to learn. The communication model consists of a central …

Federated Learning Multi-Armed Bandits

Best-of-Both-Worlds Linear Contextual Bandits

2023-12-27 · Masahiro Kato, Shinji Ito

This study investigates the problem of $K$-armed linear contextual bandits, an instance of the multi-armed bandit problem, under an adversarial corruption. At each round, a decision-maker observes an independent and iden…

Multi-Armed Bandits

Design of Experiments for Stochastic Contextual Linear Bandits

2021-07-21 · NeurIPS 2021 12 · Andrea Zanette, Kefan Dong, Jonathan Lee, Emma Brunskill

In the stochastic linear contextual bandit setting there exist several minimax procedures for exploration with policies that are reactive to the data being acquired. In practice, there can be a significant engineering ov…

Truncated LinUCB for Stochastic Linear Bandits

2022-02-23 · Yanglei Song, Meng Zhou

This paper considers contextual bandits with a finite number of arms, where the contexts are independent and identically distributed $d$-dimensional random vectors, and the expected rewards are linear in both the arm par…

Multi-Armed Bandits

Fast and Sample Efficient Multi-Task Representation Learning in Stochastic Contextual Bandits

2024-10-02 · Jiabin Lin, Shana Moothedath, Namrata Vaswani

We study how representation learning can improve the learning efficiency of contextual bandit problems. We study the setting where we play T contextual linear bandits with dimension d simultaneously, and these T bandit t…

Multi-Armed Bandits Multi-Task Learning Representation Learning