Adopting Robustness and Optimality in Fitting and Learning
We generalized a modified exponentialized estimator by pushing the robust-optimal (RO) index $\lambda$ to $-\infty$ for achieving robustness to outliers by optimizing a quasi-Minimin function. The robustness is realized and controlled adaptively by the RO index without any predefined threshold. Optimality is guaranteed by expansion of the convexity region in the Hessian matrix to largely avoid local optima. Detailed quantitative analysis on both robustness and optimality are provided. The results of proposed experiments on fitting tasks for three noisy non-convex functions and the digits recognition task on the MNIST dataset consolidate the conclusions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
I-PGD-AT: Efficient Adversarial Training via Imitating Iterative PGD Attack
Adversarial training has been widely used in various machine learning paradigms to improve the robustness; while it would increase the training cost due to the perturbation optimization process. To improve the efficiency…
Adversarial Training with Stochastic Weight Average
Adversarial training deep neural networks often experience serious overfitting problem. Recently, it is explained that the overfitting happens because the sample complexity of training data is insufficient to generalize …
Sparse L0-norm based Kernel-free Quadratic Surface Support Vector Machines
Kernel-free quadratic surface support vector machine (SVM) models have gained significant attention in machine learning. However, introducing a quadratic classifier increases the model's complexity by quadratically expan…
Computational EfficiencyOn overfitting and asymptotic bias in batch reinforcement learning with partial observability
This paper provides an analysis of the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data) in the context of reinforcement learning with par…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Volume Optimality in Conformal Prediction with Structured Prediction Sets
Conformal Prediction is a widely studied technique to construct prediction sets of future observations. Most conformal prediction methods focus on achieving the necessary coverage guarantees, but do not provide formal gu…
Conformal PredictionPredictionStructured Prediction