Convex Representation Learning for Generalized Invariance in Semi-Inner-Product Space
Invariance (defined in a general sense) has been one of the most effective priors for representation learning. Direct factorization of parametric models is feasible only for a small range of invariances, while regularization approaches, despite improved generality, lead to nonconvex optimization. In this work, we develop a convex representation learning algorithm for a variety of generalized invariances that can be modeled as semi-norms. Novel Euclidean embeddings are introduced for kernel representers in a semi-inner-product space, and approximation bounds are established. This allows invariant representations to be learned efficiently and effectively as confirmed in our experiments, along with accurate predictions.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningSimilar Papers 제목 키워드 기반
Semi-Inner-Products for Convex Functionals and Their Use in Image Decomposition
Semi-inner-products in the sense of Lumer are extended to convex functionals. This yields a Hilbert-space like structure to convex functionals in Banach spaces. In particular, a general expression for semi-inner-products…
A Generalized Metriplectic System via Free Energy and System~Identification via Bilevel Convex Optimization
This work generalizes the classical metriplectic formalism to model Hamiltonian systems with nonconservative dissipation. Classical metriplectic representations allow for the description of energy conservation and produc…
Unifying distillation and privileged information
Distillation (Hinton et al., 2015) and privileged information (Vapnik & Izmailov, 2015) are two techniques that enable machines to learn from other machines. This paper unifies these two techniques into generalized disti…
A Semi-Decentralized Tikhonov-based Algorithm for Optimal Generalized Nash Equilibrium Selection
To optimally select a generalized Nash equilibrium, in this paper, we propose a semi-decentralized algorithm based on a double-layer Tikhonov regularization method. Technically, we extend the Tikhonov method for equilibr…
Risk measures beyond frictionless markets
We develop a general theory of risk measures that determines the optimal amount of capital to raise and invest in a portfolio of reference traded securities in order to meet a pre-specified regulatory requirement. The di…
Translation