Encoding categorical data: Is there yet anything 'hotter' than one-hot encoding?
Categorical features are present in about 40% of real world problems, highlighting the crucial role of encoding as a preprocessing component. Some recent studies have reported benefits of the various target-based encoders over classical target-agnostic approaches. However, these claims are not supported by any statistical analysis, and are based on a single dataset or a very small and heterogeneous sample of datasets. The present study explores the encoding effects in an exhaustive sample of classification problems from OpenML repository. We fitted linear mixed-effects models to the experimental data, treating task ID as a random effect, and the encoding scheme and the various characteristics of categorical features as fixed effects. We found that in multiclass tasks, one-hot encoding and Helmert contrast coding outperform target-based encoders. In binary tasks, there were no significant differences across the encoding schemes; however, one-hot encoding demonstrated a marginally positive effect on the outcome. Importantly, we found no significant interactions between the encoding schemes and the characteristics of categorical features. This suggests that our findings are generalizable to a wide variety of problems across domains.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
GraphOTTER: Evolving LLM-based Graph Reasoning for Complex Table Question Answering
Complex Table Question Answering involves providing accurate answers to specific questions based on intricate tables that exhibit complex layouts and flexible header locations. Despite considerable progress having been m…
Question AnsweringUnsupervised Anomaly Detection for Auditing Data and Impact of Categorical Encodings
In this paper, we introduce the Vehicle Claims dataset, consisting of fraudulent insurance claims for automotive repairs. The data belongs to the more broad category of Auditing data, which includes also Journals and Net…
Anomaly DetectionContrastive LearningDensity EstimationUnsupervised Anomaly DetectionFairness Implications of Encoding Protected Categorical Attributes
Past research has demonstrated that the explicit use of protected attributes in machine learning can improve both performance and fairness. Many machine learning algorithms, however, cannot directly process categorical a…
FairnessFeature EngineeringHotter and Colder: A New Approach to Annotating Sentiment, Emotions, and Bias in Icelandic Blog Comments
This paper presents Hotter and Colder, a dataset designed to analyze various types of online behavior in Icelandic blog comments. Building on previous work, we used GPT-4o mini to annotate approximately 800,000 comments …
Sentiment AnalysisQuantifying impacts of the drought 2018 on European ecosystems in comparison to 2003
In recent decades, an increasing persistence of atmospheric circulation patterns has been observed. In the course of the associated long-lasting anticyclonic summer circulations, heat waves and drought spells often coinc…