paper-with-me

Papers

Second-order difference subspace

2024-09-13 · Kazuhiro Fukui, Pedro H. V. Valois, Lincon Souza, Takumi Kobayashi

Subspace representation is a fundamental technique in various fields of machine learning. Analyzing a geometrical relationship among multiple subspaces is essential for understanding subspace series' temporal and/or spatial dynamics. This paper proposes the second-order difference subspace, a higher-order extension of the first-order difference subspace between two subspaces that can analyze the geometrical difference between them. As a preliminary for that, we extend the definition of the first-order difference subspace to the more general setting that two subspaces with different dimensions have an intersection. We then define the second-order difference subspace by combining the concept of first-order difference subspace and principal component subspace (Karcher mean) between two subspaces, motivated by the second-order central difference method. We can understand that the first/second-order difference subspaces correspond to the velocity and acceleration of subspace dynamics from the viewpoint of a geodesic on a Grassmann manifold. We demonstrate the validity and naturalness of our second-order difference subspace by showing numerical results on two applications: temporal shape analysis of a 3D object and time series analysis of a biometric signal.

📄 PDF Abstract BibTeX arXiv:2409.08563

Code (0)

등록된 구현이 없습니다.

Tasks

Time Series Analysis

Similar Papers 제목 키워드 기반

Krylov Cubic Regularized Newton: A Subspace Second-Order Method with Dimension-Free Convergence Rate

2024-01-05 · Ruichen Jiang, Parameswaran Raman, Shoham Sabach, Aryan Mokhtari 외

Second-order optimization methods, such as cubic regularized Newton methods, are known for their rapid convergence rates; nevertheless, they become impractical in high-dimensional problems due to their substantial memory…

Second-order methodssubspace methods

Random Subspace Cubic-Regularization Methods, with Applications to Low-Rank Functions

2025-01-16 · Coralia Cartis, Zhen Shao, Edward Tansley

We propose and analyze random subspace variants of the second-order Adaptive Regularization using Cubics (ARC) algorithm. These methods iteratively restrict the search space to some random subspace of the parameters, con…

ARC

Subspace-based Approximate Hessian Method for Zeroth-Order Optimization

2025-07-08 · Dongyoon Kim, Sungjae Lee, Wonjin Lee, Kwang In Kim

Zeroth-order optimization addresses problems where gradient information is inaccessible or impractical to compute. While most existing methods rely on first-order approximations, incorporating second-order (curvature) in…

AdaSub: Stochastic Optimization Using Second-Order Information in Low-Dimensional Subspaces

2023-10-30 · João Victor Galvão da Mata, Martin S. Andersen

We introduce AdaSub, a stochastic optimization algorithm that computes a search direction based on second-order information in a low-dimensional subspace that is defined adaptively based on available current and past inf…

ManagementSecond-order methodsStochastic Optimization

Stochastic Subspace Cubic Newton Method

2020-02-21 · ICML 2020 1 · Filip Hanzely, Nikita Doikov, Peter Richtárik, Yurii Nesterov

In this paper, we propose a new randomized second-order optimization algorithm---Stochastic Subspace Cubic Newton (SSCN)---for minimizing a high dimensional convex function $f$. Our method can be seen both as a {\em stoc…

Second-order methods