Benchmarking Large Language Model Volatility
The impact of non-deterministic outputs from Large Language Models (LLMs) is not well examined for financial text understanding tasks. Through a compelling case study on investing in the US equity market via news sentiment analysis, we uncover substantial variability in sentence-level sentiment classification results, underscoring the innate volatility of LLM outputs. These uncertainties cascade downstream, leading to more significant variations in portfolio construction and return. While tweaking the temperature parameter in the language model decoder presents a potential remedy, it comes at the expense of stifled creativity. Similarly, while ensembling multiple outputs mitigates the effect of volatile outputs, it demands a notable computational investment. This work furnishes practitioners with invaluable insights for adeptly navigating uncertainty in the integration of LLMs into financial decision-making, particularly in scenarios dictated by non-deterministic information.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingDecision MakingDecoderLanguage ModelingLanguage ModellingLarge Language ModelmodelSentenceSentiment AnalysisSentiment ClassificationSimilar Papers 제목 키워드 기반
On Stable Long-Form Generation: Benchmarking and Mitigating Length Volatility
Large Language Models (LLMs) excel at long-context understanding but exhibit significant limitations in long-form generation. Existing studies primarily focus on single-generation quality, generally overlooking the volat…
Long-Context UnderstandingPortfolio Benchmarking under Drawdown Constraint and Stochastic Sharpe Ratio
We consider an investor who seeks to maximize her expected utility derived from her terminal wealth relative to the maximum performance achieved over a fixed time horizon, and under a portfolio drawdown constraint, in a …
BenchmarkingA Comparison of Cryptocurrency Volatility-benchmarking New and Mature Asset Classes
The paper analyzes the cryptocurrency ecosystem at both the aggregate and individual levels to understand the factors that impact future volatility. The study uses high-frequency panel data from 2020 to 2022 to examine t…
BenchmarkingBenchmarking Deep Sequential Models on Volatility Predictions for Financial Time Series
Volatility is a quantity of measurement for the price movements of stocks or options which indicates the uncertainty within financial markets. As an indicator of the level of risk or the degree of variation, volatility i…
BenchmarkingDecision MakingTime SeriesTime Series AnalysisCSPO: Cross-Market Synergistic Stock Price Movement Forecasting with Pseudo-volatility Optimization
The stock market, as a cornerstone of the financial markets, places forecasting stock price movements at the forefront of challenges in quantitative finance. Emerging learning-based approaches have made significant progr…
Benchmarking