Neural Eigenfunctions Are Structured Representation Learners

2022-10-23 · Zhijie Deng, Jiaxin Shi, Hao Zhang, Peng Cui, Cewu Lu, Jun Zhu

This paper introduces a structured, adaptive-length deep representation called Neural Eigenmap. Unlike prior spectral methods such as Laplacian Eigenmap that operate in a nonparametric manner, Neural Eigenmap leverages NeuralEF to parametrically model eigenfunctions using a neural network. We show that, when the eigenfunction is derived from positive relations in a data augmentation setup, applying NeuralEF results in an objective function that resembles those of popular self-supervised learning methods, with an additional symmetry-breaking property that leads to \emph{structured} representations where features are ordered by importance. We demonstrate using such representations as adaptive-length codes in image retrieval systems. By truncation according to feature importance, our method requires up to $16\times$ shorter representation length than leading self-supervised learning ones to achieve similar retrieval performance. We further apply our method to graph data and report strong results on a node representation learning benchmark with more than one million nodes.

📄 PDF Abstract BibTeX arXiv:2210.12637

Code (1)

thudzj/NEigenmaps 공식 구현 pytorch

Tasks

Contrastive Learning Data Augmentation Feature Importance Image RetrievalLinear-Probe ClassificationRepresentation Learning Retrieval Self-Supervised Learning Transfer Learning

Methods 이 논문이 사용한 방법론

Contrastive Learning 설명 없음

Spectral Inference Networks: Unifying Deep and Spectral Learning

2018-06-06 · ICLR 2019 5 · David Pfau, Stig Petersen, Ashish Agarwal, David G. T. Barrett 외

We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization. Spectral Inference Networks generalize Slow Feature Analysis to generic symmetric operators,…

Atari Games Bilevel Optimization Representation Learning Stochastic Optimization+1

Ensemble of Weak Spectral Total Variation Learners: a PET-CT Case Study

2025-07-11 · Anna Rosenberg, John Kennedy, Zohar Keidar, Yehoshua Y. Zeevi 외 arxiv

Solving computer vision problems through machine learning, one often encounters lack of sufficient training data. To mitigate this we propose the use of ensembles of weak learners based on spectral total-variation (STV) …

Ensemble Learning

EigenGP: Gaussian Process Models with Adaptive Eigenfunctions

2014-01-02 · Hao Peng, Yuan Qi

Gaussian processes (GPs) provide a nonparametric representation of functions. However, classical GP inference suffers from high computational cost for big data. In this paper, we propose a new Bayesian approach, EigenGP,…

Gaussian Processes

On the algebra of Koopman eigenfunctions and on some of their infinities

2026-04-23 · Zahra Monfared, Saksham Malhotra, Sekiya Hajime, Ioannis Kevrekidis 외 arxiv

For continuous-time dynamical systems with reversible trajectories, the nowhere-vanishing eigenfunctions of the Koopman operator of the system form a multiplicative group. Here, we exploit this property to accelerate the…

Geometric Considerations of a Good Dictionary for Koopman Analysis of Dynamical Systems: Cardinality, 'Primary Eigenfunction,' and Efficient Representation

2019-12-18 · Erik Bollt

Representation of a dynamical system in terms of simplifying modes is a central premise of reduced order modelling and a primary concern of the increasingly popular DMD (dynamic mode decomposition) empirical interpretati…