L3iTC at the FinLLM Challenge Task: Quantization for Financial Text Classification & Summarization
This article details our participation (L3iTC) in the FinLLM Challenge Task 2024, focusing on two key areas: Task 1, financial text classification, and Task 2, financial text summarization. To address these challenges, we fine-tuned several large language models (LLMs) to optimize performance for each task. Specifically, we used 4-bit quantization and LoRA to determine which layers of the LLMs should be trained at a lower precision. This approach not only accelerated the fine-tuning process on the training data provided by the organizers but also enabled us to run the models on low GPU memory. Our fine-tuned models achieved third place for the financial classification task with an F1-score of 0.7543 and secured sixth place in the financial summarization task on the official test datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
GPUQuantizationTask 2text-classificationText ClassificationText SummarizationSimilar Papers 제목 키워드 기반
FinLoRA: Finetuning Quantized Financial Large Language Models Using Low-Rank Adaptation
Finetuned large language models (LLMs) have shown remarkable performance in financial tasks, such as sentiment analysis and information retrieval. Due to privacy concerns, finetuning and deploying Financial LLMs (FinLLMs…
GPUInformation RetrievalQuantizationRetrieval+1A Survey of Large Language Models in Finance (FinLLMs)
Large Language Models (LLMs) have shown remarkable capabilities across a wide variety of Natural Language Processing (NLP) tasks and have attracted attention from multiple domains, including financial services. Despite t…
Named Entity Recognition (NER)Question AnsweringSentiment ClassificationSurvey+3Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications
Financial LLMs hold promise for advancing financial tasks and domain-specific applications. However, they are limited by scarce corpora, weak multimodal capabilities, and narrow evaluations, making them less suited for r…
Time SeriesFinGPT: Democratizing Internet-scale Data for Financial Large Language Models
Large language models (LLMs) have demonstrated remarkable proficiency in understanding and generating human-like texts, which may potentially revolutionize the finance industry. However, existing LLMs often fall short in…
Algorithmic TradingSentiment AnalysisCatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications
The integration of Large Language Models (LLMs) into financial analysis has garnered significant attention in the NLP community. This paper presents our solution to IJCAI-2024 FinLLM challenge, investigating the capabili…
Decision MakingFinancial Analysisparameter-efficient fine-tuningTask 2+1