Proper-Composite Loss Functions in Arbitrary Dimensions
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules via the proper-composite representation, in which predictions are mapped to probability distributions which are then scored via a scoring rule. However, recent research so far has primarily been concerned with analysing the (typically) finite-dimensional conditional risk problem on the output space, leaving aside the larger total risk minimisation. We generalise a number of these results to an infinite dimensional setting and in doing so we are able to exploit the familial resemblance of density and conditional density estimation to provide a simple characterisation of the canonical link.
Code (0)
등록된 구현이 없습니다.
Tasks
Density Estimationscoring ruleSimilar Papers 제목 키워드 기반
Composite Multiclass Losses
We consider loss functions for multiclass prediction problems. We show when a multiclass loss can be expressed as a ``proper composite loss'', which is the composition of a proper loss and a link function. We exte…
General ClassificationSensitivityDeep Neural Networks and Finite Elements of Any Order on Arbitrary Dimensions
In this study, we establish that deep neural networks employing ReLU and ReLU$^2$ activation functions can effectively represent Lagrange finite element functions of any order on various simplicial meshes in arbitrary di…
Variable Selection with the Knockoffs: Composite Null Hypotheses
The fixed-X knockoff filter is a flexible framework for variable selection with false discovery rate (FDR) control in linear models with arbitrary design matrices (of full column rank) and it allows for finite-sample sel…
Variable SelectionGeneralizing Cross Entropy Loss with a Beta Proper Composite Loss: An Improved Loss Function for Open Set Recognition
Open set recognition involves identifying data instances encountered during test time that do not belong to known classes in the training set. The majority of recent deep learning approaches to open set recognition use a…
Open Set LearningStochastic Gradient Descent for Stochastic Doubly-Nonconvex Composite Optimization
The stochastic gradient descent has been widely used for solving composite optimization problems in big data analyses. Many algorithms and convergence properties have been developed. The composite functions were convex p…