Online Learning for Adversaries with Memory: Price of Past Mistakes
The framework of online learning with memory naturally captures learning problems with temporal effects, and was previously studied for the experts setting. In this work we extend the notion of learning with memory to the general Online Convex Optimization (OCO) framework, and present two algorithms that attain low regret. The first algorithm applies to Lipschitz continuous loss functions, obtaining optimal regret bounds for both convex and strongly convex losses. The second algorithm attains the optimal regret bounds and applies more broadly to convex losses without requiring Lipschitz continuity, yet is more complicated to implement. We complement the theoretic results with two applications: statistical arbitrage in finance, and multi-step ahead prediction in statistics.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Online Selective Classification with Limited Feedback
Motivated by applications to resource-limited and safety-critical domains, we study selective classification in the online learning model, wherein a predictor may abstain from classifying an instance. For example, this m…
ClassificationvalidLearning to Recover from Failures using Memory
Learning from past mistakes is a quintessential aspect of intelligence. In sequential decision-making, existing meta-learning methods that learn a learning algorithm utilize experience from only a few previous episodes t…
Decision MakingMeta-LearningSequential Decision MakingEstimating real-world probabilities: A forward-looking behavioral framework
We show that disentangling sentiment-induced biases from fundamental expectations significantly improves the accuracy and consistency of probabilistic forecasts. Using data from 1994 to 2017, we analyze 15 stochastic mod…
Learning to repair: Repairing model output errors after deployment using a dynamic memory of feedback
Large language models (LMs), while powerful, are not immune to mistakes, but can be difficult to retrain. Our goal is for an LM to continue to improve after deployment, without retraining, using feedback from the user. O…
Script GenerationR^2-Mem: Reflective Experience for Memory Search
Deep search has recently emerged as a promising paradigm for enabling agents to retrieve fine-grained historical information without heavy memory pre-managed. However, existing deep search agents for memory system repeat…