The Shooting Regressor; Randomized Gradient-Based Ensembles
An ensemble method is introduced that utilizes randomization and loss function gradients to compute a prediction. Multiple weakly-correlated estimators approximate the gradient at randomly sampled points on the error surface and are aggregated into a final solution. A scaling parameter is described that controls a trade-off between ensemble correlation and precision. Numerical methods for estimating optimal values of the parameter are described. Empirical results are computed over a popular dataset. Inferential statistics on these results show that the method is capable of outperforming existing techniques in terms of increased accuracy.
Code (1)
Similar Papers 제목 키워드 기반
Proof of AutoML: SDN based Secure Energy Trading with Blockchain in Disaster Case
In disaster scenarios where conventional energy infrastructure is compromised, secure and traceable energy trading between solar-powered households and mobile charging units becomes a necessity. To ensure the integrity o…
A Curious Case of Remarkable Resilience to Gradient Attacks via Fully Convolutional and Differentiable Front End with a Skip Connection
We tested front-end enhanced neural models where a frozen classifier was prepended by a differentiable and fully convolutional model with a skip connection. By training them using a small learning rate for about one epoc…
Adversarial RobustnessStandard errors when a regressor is randomly assigned
We examine asymptotic properties of the OLS estimator when the values of the regressor of interest are assigned randomly and independently of other regressors. We find that the OLS variance formula in this case is often …
validTheoretical Limitations of Ensembles in the Age of Overparameterization
Classic tree-based ensembles generalize better than any single decision tree. In contrast, recent empirical studies find that modern ensembles of (overparameterized) neural networks may not provide any inherent generaliz…
An RKHS Perspective on Tree Ensembles
Random Forests and Gradient Boosting are among the most effective algorithms for supervised learning on tabular data. Both belong to the class of tree-based ensemble methods, where predictions are obtained by aggregating…
Ensemble Learning