Multi-dimensional data refining strategy for effective fine-tuning LLMs
Data is a cornerstone for fine-tuning large language models, yet acquiring suitable data remains challenging. Challenges encompassed data scarcity, linguistic diversity, and domain-specific content. This paper presents lessons learned while crawling and refining data tailored for fine-tuning Vietnamese language models. Crafting such a dataset, while accounting for linguistic intricacies and striking a balance between inclusivity and accuracy, demands meticulous planning. Our paper presents a multidimensional strategy including leveraging existing datasets in the English language and developing customized data-crawling scripts with the assistance of generative AI tools. A fine-tuned LLM model for the Vietnamese language, which was produced using resultant datasets, demonstrated good performance while generating Vietnamese news articles from prompts. The study offers practical solutions and guidance for future fine-tuning models in languages like Vietnamese.
Code (0)
등록된 구현이 없습니다.
Tasks
ArticlesDiversityVietnamese Language ModelsSimilar Papers 제목 키워드 기반
Diffusion Model for Interest Refinement in Multi-Interest Recommendation
Multi-interest candidate matching plays a pivotal role in personalized recommender systems, as it captures diverse user interests from their historical behaviors. Most existing methods utilize attention mechanisms to gen…
Recommendation SystemsDLScanner: A parameter space scanner package assisted by deep learning methods
In this paper, we introduce a scanner package enhanced by deep learning (DL) techniques. The proposed package addresses two significant challenges associated with previously developed DL-based methods: slow convergence i…
Adaptive Multi-Agent Response Refinement in Conversational Systems
Large Language Models (LLMs) have demonstrated remarkable success in conversational systems by generating human-like responses. However, they can fall short, especially when required to account for personalization or spe…
Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inference
We propose a Multiscale Invertible Generative Network (MsIGN) and associated training algorithm that leverages multiscale structure to solve high-dimensional Bayesian inference. To address the curse of dimensionality, Ms…
Bayesian InferenceImage GenerationVocal Bursts Intensity PredictionMultivariate Pair Trading by Volatility & Model Adaption Trade-off
Pair trading is one of the most discussed topics among financial researches. Despite a growing base of work, portfolio management for multivariate time series is rarely discussed. On the other hand, most researches focus…
ManagementPAIR TRADINGTime SeriesTime Series Analysis