Reinforcement Learning for LLM-based Event Forecasting
We use Group Relative Policy Optimization (GRPO), a recently devised sample and memory efficient reinforcement learning method, to finetune pretrained LLMs in the range of 1.5B to 14B parameters equipped with the ability to get current information through the use of a Wikipedia revisions tool, or news summaries, to forecast real events beyond the knowledge cutoff of the LLM, as well as problems made to simulate different aspects of the dynamics of that training. We use the results of these experiments to comment on the scaling capability of LLMs for forecasting, as well as classify how judgmental forecasting fits into the verifiable/unverifiable domain taxonomy, considering the impact of the inherent aleatoric uncertainty when forecasting future events (e.g. the roll of a die). As a result of the GRPO training, we manage to bring a 1.5B parameter transformer (Qwen 2.5 1.5B) to forecasting performance superior to Claude Sonnet 3.5 over the same dataset as measured by cross entropy from the market agreed probabilities. We also discuss various dead ends on the path to this result.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
Advancing Event Forecasting through Massive Training of Large Language Models: Challenges, Solutions, and Broader Impacts
Many recent papers have studied the development of superforecaster-level event forecasting LLMs. While methodological problems with early studies cast doubt on the use of LLMs for event forecasting, recent studies with i…
Reinforcement LearningSpatiotemporal Forecasting as Planning: A Model-Based Reinforcement Learning Approach with Generative World Models
To address the dual challenges of inherent stochasticity and non-differentiable metrics in physical spatiotemporal forecasting, we propose Spatiotemporal Forecasting as Planning (SFP), a new paradigm grounded in Model-Ba…
Reinforcement LearningScaling Open-Ended Reasoning to Predict the Future
High-stakes decision making involves reasoning under uncertainty about the future. In this work, we train language models to make predictions on open-ended forecasting questions. To scale up training data, we synthesize …
Reinforcement LearningDecision MakingDirectly Forecasting Belief for Reinforcement Learning with Delays
Reinforcement learning (RL) with delays is challenging as sensory perceptions lag behind the actual events: the RL agent needs to estimate the real state of its environment based on past observations. State-of-the-art (S…
D4RLMuJoCoreinforcement-learningReinforcement Learning+1Implementing Reinforcement Learning Algorithms in Retail Supply Chains with OpenAI Gym Toolkit
From cutting costs to improving customer experience, forecasting is the crux of retail supply chain management (SCM) and the key to better supply chain performance. Several retailers are using AI/ML models to gather data…
Demand ForecastingManagementOpenAI Gymreinforcement-learning+1