paper-with-me

홈 › Papers

Bi-Lipschitz Autoencoder With Injectivity Guarantee

2026-04-08 · Qipeng Zhan, Zhuoping Zhou, Zexuan Wang, Qi Long, Li Shen arxiv

Autoencoders are widely used for dimensionality reduction, based on the assumption that high-dimensional data lies on low-dimensional manifolds. Regularized autoencoders aim to preserve manifold geometry during dimensionality reduction, but existing approaches often suffer from non-injective mappings and overly rigid constraints that limit their effectiveness and robustness. In this work, we identify encoder non-injectivity as a core bottleneck that leads to poor convergence and distorted latent representations. To ensure robustness across data distributions, we formalize the concept of admissible regularization and provide sufficient conditions for its satisfaction. In this work, we propose the Bi-Lipschitz Autoencoder (BLAE), which introduces two key innovations: (1) an injective regularization scheme based on a separation criterion to eliminate pathological local minima, and (2) a bi-Lipschitz relaxation that preserves geometry and exhibits robustness to data distribution drift. Empirical results on diverse datasets show that BLAE consistently outperforms existing methods in preserving manifold structure while remaining resilient to sampling sparsity and distribution shifts. Code is available at https://github.com/qipengz/BLAE.

📄 PDF Abstract BibTeX arXiv:2604.06701

Code (0)

등록된 구현이 없습니다.

Tasks

Dimensionality Reduction

Similar Papers 제목 키워드 기반

On the (Non) Injectivity of Piecewise Linear Janossy Pooling

2025-05-26 · Ilai Reshef, Nadav Dym

Multiset functions, which are functions that map multisets to vectors, are a fundamental tool in the construction of neural networks for multisets and graphs. To guarantee that the vector representation of the multiset i…

Quantitative Bounds for Sorting-Based Permutation-Invariant Embeddings

2025-10-25 · Nadav Dym, Matthias Wellershoff, Efstratios Tsoukanis, Daniel Levy 외 arxiv

We study permutation-invariant embeddings of $d$-dimensional point sets, which are defined by sorting $D$ independent one-dimensional projections of the input. Such embeddings arise in graph deep learning where outputs s…

Globally Injective ReLU Networks

2020-06-15 · Michael Puthawala, Konik Kothari, Matti Lassas, Ivan Dokmanić 외

Injectivity plays an important role in generative models where it enables inference; in inverse problems and compressed sensing with generative priors it is a precursor to well posedness. We establish sharp characterizat…

compressed sensing

Transformer Injectivity & Geometric Robustness - Analytic Margins and Bi-Lipschitz Uniformity of Sequence-Level Hidden States

2025-11-17 · Mikael von Strauss arxiv

Under real-analytic assumptions on decoder-only Transformers, recent work shows that the map from discrete prompts to last-token hidden states is generically injective on finite prompt sets. We refine this picture: for e…

Optimal lower Lipschitz bounds for ReLU layers, saturation, and phase retrieval

2025-02-14 · Daniel Freeman, Daniel Haider

The injectivity of ReLU layers in neural networks, the recovery of vectors from clipped or saturated measurements, and (real) phase retrieval in $\mathbb{R}^n$ allow for a similar problem formulation and characterization…

Retrieval