Multitask Online Learning: Listen to the Neighborhood Buzz
We study multitask online learning in a setting where agents can only exchange information with their neighbors on an arbitrary communication network. We introduce $\texttt{MT-CO}_2\texttt{OL}$, a decentralized algorithm for this setting whose regret depends on the interplay between the task similarities and the network structure. Our analysis shows that the regret of $\texttt{MT-CO}_2\texttt{OL}$ is never worse (up to constants) than the bound obtained when agents do not share information. On the other hand, our bounds significantly improve when neighboring agents operate on similar tasks. In addition, we prove that our algorithm can be made differentially private with a negligible impact on the regret. Finally, we provide experimental support for our theory.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference
Large language models (LLMs) are essential in natural language processing but often struggle with inference speed and computational efficiency, limiting real-time deployment. The key-value (KV) cache mechanism reduces co…
Computational EfficiencyQuestion AnsweringText SummarizationA Coevolutionary Variable Neighborhood Search Algorithm for Discrete Multitasking (CoVNS): Application to Community Detection over Graphs
The main goal of the multitasking optimization paradigm is to solve multiple and concurrent optimization tasks in a simultaneous way through a single search process. For attaining promising results, potential complementa…
Community DetectionCan Large Language Models Understand Internet Buzzwords Through User-Generated Content
The massive user-generated content (UGC) available in Chinese social media is giving rise to the possibility of studying internet buzzwords. In this paper, we study if large language models (LLMs) can generate accurate d…
Multitask Training with Text Data for End-to-End Speech Recognition
We propose a multitask training method for attention-based end-to-end speech recognition models. We regularize the decoder in a listen, attend, and spell model by multitask training it on both audio-text and text-only da…
DecoderLanguage ModelingLanguage Modellingspeech-recognition+1LiSTEN: Learning Soft Token Embeddings for Neural Audio LLMs
Foundation models based on large language models (LLMs) have shown great success in handling various tasks and modalities. However, adapting these models for general-purpose audio-language tasks is challenging due to dif…
Diversity