SMARTFinRAG: Interactive Modularized Financial RAG Benchmark
Financial sectors are rapidly adopting language model technologies, yet evaluating specialized RAG systems in this domain remains challenging. This paper introduces SMARTFinRAG, addressing three critical gaps in financial RAG assessment: (1) a fully modular architecture where components can be dynamically interchanged during runtime; (2) a document-centric evaluation paradigm generating domain-specific QA pairs from newly ingested financial documents; and (3) an intuitive interface bridging research-implementation divides. Our evaluation quantifies both retrieval efficacy and response quality, revealing significant performance variations across configurations. The platform's open-source architecture supports transparent, reproducible research while addressing practical deployment challenges faced by financial institutions implementing RAG systems.
Code (1)
Tasks
Language ModelingLanguage ModellingRAGRetrievalMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Creating Scalable and Interactive Web Applications Using High Performance Latent Variable Models
In this project we outline a modularized, scalable system for comparing Amazon products in an interactive and informative way using efficient latent variable models and dynamic visualization. We demonstrate how our syste…
FinTruthQA: A Benchmark Dataset for Evaluating the Quality of Financial Information Disclosure
Accurate and transparent financial information disclosure is crucial in the fields of accounting and finance, ensuring market efficiency and investor confidence. Among many information disclosure platforms, the Chinese s…
Language ModellingLarge Language ModelMSPM: A Modularized and Scalable Multi-Agent Reinforcement Learning-based System for Financial Portfolio Management
Financial portfolio management (PM) is one of the most applicable problems in reinforcement learning (RL) owing to its sequential decision-making nature. However, existing RL-based approaches rarely focus on scalability …
Decision MakingManagementMulti-agent Reinforcement LearningPortfolio Optimization+3Tianshou: a Highly Modularized Deep Reinforcement Learning Library
In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend. Tianshou intends to be research-friendly by providing a flexible and reliabl…
Deep Reinforcement LearningMuJoCoreinforcement-learningReinforcement Learning+1WrAFT: a Modularized Automated Writing Evaluation System for Argumentative Essays
This study presents WrAFT, a Writing Assessment and Feedback Tool, that delivers both accurate and reliable scores and effective comprehensive feedback to argumentative essays. WrAFT adopts a modular design by dividing a…