Muon on the Stiefel Manifold Admits an Exact Closed-Form Update
We study Muon, a recently proposed matrix-aware optimization method, in the context of the Stiefel manifold. This manifold consists of matrices with orthonormal columns and is ubiquitous in machine learning and scientific computing. Existing extensions of Muon to this manifold rely on heuristic, approximate, or iterative updates with varying computational efficiency. We show that the corresponding Stiefel Muon update admits an exact closed-form solution and use this result to develop Skewon, a practical algorithm for orthogonality-constrained optimization with an efficient implementation. We further establish first-order convergence guarantees for Skewon in the smooth non-convex setting.
Code (0)
등록된 구현이 없습니다.
Tasks
Computational EfficiencySimilar Papers 제목 키워드 기반
Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds
Muon and related norm-constrained matrix optimizers have become central to large-scale learning problems. They are formulated as a linear maximization oracle (LMO) over an ambient matrix-norm ball in unconstrained Euclid…
Image ClassificationManifold constrained steepest descent for smooth and closed-set optimization
We study minimization of smooth functions over feasible sets that have smooth embedded-manifold structure throughout or only on selected regions, using linear minimization oracles (LMOs) to determine search directions un…
Muon Converges under Heavy-Tailed Noise: Nonconvex Hölder-Smooth Empirical Risk Minimization
Muon is a recently proposed optimizer that enforces orthogonality in parameter updates by projecting gradients onto the Stiefel manifold, leading to stable and efficient training in large-scale deep neural networks. Mean…
ManifoldFlow: SPD-Relaxed Stiefel Layers with Learnable Singular Spectrum
Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonorma…
Closed-form geodesics and trust-region method to calculate Riemannian logarithms on Stiefel and its quotient manifolds
We provide two closed-form geodesic formulas for a family of metrics on Stiefel manifold, parameterized by two positive numbers, having both the embedded and canonical metrics as special cases. The closed-form formulas a…
Form