LLM4FS: Leveraging Large Language Models for Feature Selection and How to Improve It
Recent advances in large language models (LLMs) have provided new opportunities for decision-making, particularly in the task of automated feature selection. In this paper, we first comprehensively evaluate LLM-based feature selection methods, covering the state-of-the-art DeepSeek-R1, GPT-o3-mini, and GPT-4.5. Then, we propose a novel hybrid strategy called LLM4FS that integrates LLMs with traditional data-driven methods. Specifically, input data samples into LLMs, and directly call traditional data-driven techniques such as random forest and forward sequential selection. Notably, our analysis reveals that the hybrid strategy leverages the contextual understanding of LLMs and the high statistical reliability of traditional data-driven methods to achieve excellent feature selection performance, even surpassing LLMs and traditional data-driven methods. Finally, we point out the limitations of its application in decision-making.
Code (0)
등록된 구현이 없습니다.
Tasks
Decision Makingfeature selectionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Leveraging Encoder-only Large Language Models for Mobile App Review Feature Extraction
Mobile app review analysis presents unique challenges due to the low quality, subjective bias, and noisy content of user-generated documents. Extracting features from these reviews is essential for tasks such as feature …
Sentiment Analysistoken-classificationToken ClassificationExploring Large Language Models for Feature Selection: A Data-centric Perspective
The rapid advancement of Large Language Models (LLMs) has significantly influenced various domains, leveraging their exceptional few-shot and zero-shot learning capabilities. In this work, we aim to explore and understan…
Descriptivefeature selectionZero-Shot LearningPre-training via Leveraging Assisting Languages and Data Selection for Neural Machine Translation
Sequence-to-sequence (S2S) pre-training using large monolingual data is known to improve performance for various S2S NLP tasks in low-resource settings. However, large monolingual corpora might not always be available fo…
Machine TranslationNMTTranslationFrom Insight to Action: A Novel Framework for Interpretability-Guided Data Selection in Large Language Models
While mechanistic interpretability tools like Sparse Autoencoders (SAEs) can uncover meaningful features within Large Language Models (LLMs), a critical gap remains in transforming these insights into practical actions f…
Mathematical ReasoningMalware Classification Leveraging NLP & Machine Learning for Enhanced Accuracy
This paper investigates the application of natural language processing (NLP)-based n-gram analysis and machine learning techniques to enhance malware classification. We explore how NLP can be used to extract and analyze …
Classificationfeature selectionMalware Classification