Evaluating K-Fold Cross Validation for Transformer Based Symbolic Regression Models
Symbolic Regression remains an NP-Hard problem, with extensive research focusing on AI models for this task. Transformer models have shown promise in Symbolic Regression, but performance suffers with smaller datasets. We propose applying k-fold cross-validation to a transformer-based symbolic regression model trained on a significantly reduced dataset (15,000 data points, down from 500,000). This technique partitions the training data into multiple subsets (folds), iteratively training on some while validating on others. Our aim is to provide an estimate of model generalization and mitigate overfitting issues associated with smaller datasets. Results show that this process improves the model's output consistency and generalization by a relative improvement in validation loss of 53.31%. Potentially enabling more efficient and accessible symbolic regression in resource-constrained environments.
Code (0)
등록된 구현이 없습니다.
Tasks
regressionSymbolic RegressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
From Theory to Practice: Implementing and Evaluating e-Fold Cross-Validation
This paper introduces e-fold cross-validation, an energy-efficient alternative to k-fold cross-validation. It dynamically adjusts the number of folds based on a stopping criterion. The criterion checks after each fold wh…
SiNFluD: Creating and Evaluating Figurative Language Dataset for Sindhi
In this article, we introduce SiNFluD, a novel benchmark dataset for Sindhi figurative language classification. We first collect raw text from various blogs, social media platforms, and literary sources, and subsequently…
A Task-Oriented Dialogue Architecture via Transformer Neural Language Models and Symbolic Injection
Recently, transformer language models have been applied to build both task- and non-task-oriented dialogue systems. Although transformers perform well on most of the NLP tasks, they perform poorly on context retrieval an…
Dialogue ManagementManagementRetrievalTask-Oriented Dialogue Systems+1Context-aware Skin Cancer Epithelial Cell Classification with Scalable Graph Transformers
Whole-slide images (WSIs) from cancer patients contain rich information that can be used for medical diagnosis or to follow treatment progress. To automate their analysis, numerous deep learning methods based on convolut…
Medical DiagnosisEmergent Ordinal Geometry in Transformers Trained on Local Comparisons
Transitive inference is the challenge of inferring that A < C from knowing only adjacent relations (A < B, B < C). It is solved by humans and animals not through logical chaining but via an analogue mental number line, w…