Scoring rule nets: beyond mean target prediction in multivariate regression
Probabilistic regression models trained with maximum likelihood estimation (MLE), can sometimes overestimate variance to an unacceptable degree. This is mostly problematic in the multivariate domain. While univariate models often optimize the popular Continuous Ranked Probability Score (CRPS), in the multivariate domain, no such alternative to MLE has yet been widely accepted. The Energy Score - the most investigated alternative - notoriously lacks closed-form expressions and sensitivity to the correlation between target variables. In this paper, we propose Conditional CRPS: a multivariate strictly proper scoring rule that extends CRPS. We show that closed-form expressions exist for popular distributions and illustrate their sensitivity to correlation. We then show in a variety of experiments on both synthetic and real data, that Conditional CRPS often outperforms MLE, and produces results comparable to state-of-the-art non-parametric models, such as Distributional Random Forest (DRF).
Code (0)
등록된 구현이 없습니다.
Tasks
Formregressionscoring ruleSensitivitySimilar Papers 제목 키워드 기반
Generative Adversarial Nets for Robust Scatter Estimation: A Proper Scoring Rule Perspective
Robust scatter estimation is a fundamental task in statistics. The recent discovery on the connection between robust estimation and generative adversarial nets (GANs) by Gao et al. (2018) suggests that it is possible to …
scoring ruleFields of the Planet: Field Boundary Mapping Beyond 10m
Field-boundary maps support crop monitoring, irrigation planning, and yield estimation, but many smallholder parcels span only a few 10 m Sentinel-2 pixels. We introduce Fields of the Planet (FTP), a 3 m PlanetScope comp…
Choice of Scoring Rules for Indirect Elicitation of Properties with Parametric Assumptions
People are commonly interested in predicting a statistical property of a random event such as mean and variance. Proper scoring rules assess the quality of predictions and require that the expected score gets uniquely ma…
Aligned Textual Scoring Rules
Scoring rules elicit probabilistic predictions from a strategic agent by scoring the prediction against a ground truth state. A scoring rule is proper if, from the agent's perspective, reporting the true belief maximizes…
scoring ruleOptimizing Rescoring Rules with Interpretable Representations of Long-Term Information
Analyzing temporal data (e.g., wearable device data) requires a decision about how to combine information from the recent and distant past. In the context of classifying sleep status from actigraphy, Webster's rescoring …
Binary Classification