Selectivity Estimation for Range Predicates using Lightweight Models
Query optimizers depend on selectivity estimates of query predicates to produce a good execution plan. When a query contains multiple predicates, today’s optimizers use a variety of assumptions, such as independence between predicates, to estimate selectivity. While such techniques have the benefit of fast estimation and small memory footprint, they often incur large selectivity estimation errors. In this work, we reconsider selectivity estimation as a regression problem. We explore application of neural networks and tree-based ensembles to the important problem of selectivity estimation of multi-dimensional range predicates. While their straightforward application does not outperform even simple baselines, we propose two simple yet effective design choices, i.e., regression label transformation and feature engineering, motivated by the selectivity estimation context. Through extensive empirical evaluation across a variety of datasets, we show that the proposed models deliver both highly accurate estimates as well as fast estimation.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature EngineeringregressionSimilar Papers 제목 키워드 기반
Multi-Attribute Selectivity Estimation Using Deep Learning
Selectivity estimation - the problem of estimating the result size of queries - is a fundamental problem in databases. Accurate estimation of query selectivity involving multiple correlated attributes is especially chall…
AttributeDeep LearningDensity EstimationSelectivity Estimation for Linear Queries via Online Learning
Learning-based approaches for selectivity estimation in databases have gained significant traction in recent years. However, theoretical studies of these learning-based approaches are essentially limited to fixed query d…
Compositional Online Learning for Semantic Data Processing Systems
An LLM call in a semantic data processing system is expensive enough to dominate query cost, yet slow enough to hide a CPU-side learner's update behind its round-trip. In production, LLM compute accounts for $80-90\%$ of…
Consistent and Flexible Selectivity Estimation for High-Dimensional Data
Selectivity estimation aims at estimating the number of database objects that satisfy a selection criterion. Answering this problem accurately and efficiently is essential to many applications, such as density estimation…
Data IntegrationDensity EstimationOutlier DetectionVocal Bursts Intensity PredictionMLSkip: Data Skipping for ML Filters via Lightweight Metadata
Database vendors recently released AI functions that can be used in filter predicates. As such functions often rely on costly, black-box ML models, they unveil new data management challenges. Concretely, traditional data…