Quantile Encoder: Tackling High Cardinality Categorical Features in Regression Problems
Regression problems have been widely studied in machinelearning literature resulting in a plethora of regression models and performance measures. However, there are few techniques specially dedicated to solve the problem of how to incorporate categorical features to regression problems. Usually, categorical feature encoders are general enough to cover both classification and regression problems. This lack of specificity results in underperforming regression models. In this paper,we provide an in-depth analysis of how to tackle high cardinality categor-ical features with the quantile. Our proposal outperforms state-of-the-encoders, including the traditional statistical mean target encoder, when considering the Mean Absolute Error, especially in the presence of long-tailed or skewed distributions. Besides, to deal with possible overfitting when there are categories with small support, our encoder benefits from additive smoothing. Finally, we describe how to expand the encoded values by creating a set of features with different quantiles. This expanded encoder provides a more informative output about the categorical feature in question, further boosting the performance of the regression model.
Code (2)
Tasks
regressionSpecificityVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
CardiCat: a Variational Autoencoder for High-Cardinality Tabular Data
High-cardinality categorical features are a common characteristic of mixed-type tabular datasets. Existing generative model architectures struggle to learn the complexities of such data at scale, primarily due to the dif…
DecoderEncoding high-cardinality string categorical variables
Statistical models usually require vector representations of categorical variables, using for instance one-hot encoding. This strategy breaks down when the number of categories grows, as it creates high-dimensional featu…
AutoMLFeature EngineeringVocal Bursts Intensity PredictionUsing Random Effects to Account for High-Cardinality Categorical Features and Repeated Measures in Deep Neural Networks
High-cardinality categorical features are a major challenge for machine learning methods in general and for deep learning in particular. Existing solutions such as one-hot encoding and entity embeddings can be hard to sc…
Entity EmbeddingsNew PCA-based Category Encoder for Cybersecurity and Processing Data in IoT Devices
Increasing the cardinality of categorical variables might decrease the overall performance of machine learning (ML) algorithms. This paper presents a novel computational preprocessing method to convert categorical to num…
Binary ClassificationA Comparison of Machine Learning Methods for Data with High-Cardinality Categorical Variables
High-cardinality categorical variables are variables for which the number of different levels is large relative to the sample size of a data set, or in other words, there are few data points per level. Machine learning m…