Data Management For Training Large Language Models: A Survey
Data plays a fundamental role in training Large Language Models (LLMs). Efficient data management, particularly in formulating a well-suited training dataset, is significant for enhancing model performance and improving training efficiency during pretraining and supervised fine-tuning stages. Despite the considerable importance of data management, the underlying mechanism of current prominent practices are still unknown. Consequently, the exploration of data management has attracted more and more attention among the research community. This survey aims to provide a comprehensive overview of current research in data management within both the pretraining and supervised fine-tuning stages of LLMs, covering various aspects of data management strategy design. Looking into the future, we extrapolate existing challenges and outline promising directions for development in this field. Therefore, this survey serves as a guiding resource for practitioners aspiring to construct powerful LLMs through efficient data management practices. The collection of the latest papers is available at https://github.com/ZigeW/data_management_LLM.
Code (1)
Tasks
ManagementSurveySimilar Papers 제목 키워드 기반
Harnessing Large Language Models for Disaster Management: A Survey
Large language models (LLMs) have revolutionized scientific research with their exceptional capabilities and transformed various fields. Among their practical applications, LLMs have been playing a crucial role in mitiga…
ManagementSurveyA Survey on Large Language Model Acceleration based on KV Cache Management
Large Language Models (LLMs) have revolutionized a wide range of domains such as natural language processing, computer vision, and multi-modal tasks due to their ability to comprehend context and perform logical reasonin…
Language ModelingLanguage ModellingLarge Language ModelLogical Reasoning+3Towards a Neural Era in Dialogue Management for Collaboration: A Literature Survey
Dialogue-based human-AI collaboration can revolutionize collaborative problem-solving, creative exploration, and social support. To realize this goal, the development of automated agents proficient in skills such as nego…
Dialogue ManagementManagementSurveyA Survey on Large Language Models for Communication, Network, and Service Management: Application Insights, Challenges, and Future Directions
The rapid evolution of communication networks in recent decades has intensified the need for advanced Network and Service Management (NSM) strategies to address the growing demands for efficiency, scalability, enhanced p…
ManagementSurveyA Survey of LLM $\times$ DATA
The integration of large language model (LLM) and data management (DATA) is rapidly redefining both domains. In this survey, we comprehensively review the bidirectional relationships. On the one hand, DATA4LLM, spanning …
Large Language ModelManagementRAGRetrieval+2