Target Variable Engineering
How does the formulation of a target variable affect performance within the ML pipeline? The experiments in this study examine numeric targets that have been binarized by comparing against a threshold. We compare the predictive performance of regression models trained to predict the numeric targets vs. classifiers trained to predict their binarized counterparts. Specifically, we make this comparison at every point of a randomized hyperparameter optimization search to understand the effect of computational resource budget on the tradeoff between the two. We find that regression requires significantly more computational effort to converge upon the optimal performance, and is more sensitive to both randomness and heuristic choices in the training process. Although classification can and does benefit from systematic hyperparameter tuning and model selection, the improvements are much less than for regression. This work comprises the first systematic comparison of regression and classification within the framework of computational resource requirements. Our findings contribute to calls for greater replicability and efficiency within the ML pipeline for the sake of building more sustainable and robust AI systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Hyperparameter OptimizationModel SelectionregressionSimilar Papers 제목 키워드 기반
Causal Information Splitting: Engineering Proxy Features for Robustness to Distribution Shifts
Statistical prediction models are often trained on data from different probability distributions than their eventual use cases. One approach to proactively prepare for these shifts harnesses the intuition that causal mec…
counterfactualfeature selectionMulti-level Diagnosis and Evaluation for Robust Tabular Feature Engineering with Large Language Models
Recent advancements in large language models (LLMs) have shown promise in feature engineering for tabular data, but concerns about their reliability persist, especially due to variability in generated outputs. We introdu…
Feature EngineeringAssets Forecasting with Feature Engineering and Transformation Methods for LightGBM
Fluctuations in the stock market rapidly shape the economic world and consumer markets, impacting millions of individuals. Hence, accurately forecasting it is essential for mitigating risks, including those associated wi…
Feature EngineeringFeature ImportanceIdentifying biological perturbation targets through causal differential networks
Identifying variables responsible for changes to a biological system enables applications in drug target discovery and cell engineering. Given a pair of observational and interventional datasets, the goal is to isolate t…
Causal DiscoveryProcessBERT: Towards Equivalence Judgment of Variable Definitions among Multiple Engineering Documents
Physical models play an important role in the process industry. However, conventional physical model building requires a survey on a huge amount of literature and trial-and-error to improve the model performance. We aim …
Language ModelingLanguage Modelling