Modified online Newton step based on element wise multiplication
The second order method as Newton Step is a suitable technique in Online Learning to guarantee regret bound. The large data is a challenge in Newton method to store second order matrices as hessian. In this paper, we have proposed an modified online Newton step that store first and second order matrices of dimension m (classes) by d (features). we have used element wise arithmetic operation to retain matrices size same. The modified second order matrix size results in faster computations. Also, the mistake rate is at par with respect to popular methods in literature. The experiments outcome indicate that proposed method could be helpful to handle large multi class datasets in common desktop machines using second order method as Newton step.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Valuation of the Convertible Bonds under Penalty TF model using Finite Element Method
In this paper, the TF system of two-coupled Black-Scholes equations for pricing the convertible bonds is solved numerically by using the P1 and P2 finite elements with the inequality constraints approximated by the penal…
Improving Pseudo-Time Stepping Convergence for CFD Simulations With Neural Networks
Computational fluid dynamics (CFD) simulations of viscous fluids described by the Navier-Stokes equations are considered. Depending on the Reynolds number of the flow, the Navier-Stokes equations may exhibit a highly non…
NeST-BO: Fast Local Bayesian Optimization via Newton-Step Targeting of Gradient and Hessian Information
Bayesian optimization (BO) is effective for expensive black-box problems but remains challenging in high dimensions. We propose NeST-BO, a curvature-aware local BO method that targets a (modified) Newton step by jointly …
SNAP: A semismooth Newton algorithm for pathwise optimization with optimal local convergence rate and oracle properties
We propose a semismooth Newton algorithm for pathwise optimization (SNAP) for the LASSO and Enet in sparse, high-dimensional linear regression. SNAP is derived from a suitable formulation of the KKT conditions based on N…
regressionA fast neural hybrid Newton solver adapted to implicit methods for nonlinear dynamics
The use of implicit time-stepping schemes for the numerical approximation of solutions to stiff nonlinear time-evolution equations brings well-known advantages including, typically, better stability behaviour and corresp…
Operator learning