paper-with-me

홈 › Papers

Generalizing Nonlinear ICA Beyond Structural Sparsity

2023-11-01 · NeurIPS 2023 11

Nonlinear independent component analysis (ICA) aims to uncover the true latent sources from their observable nonlinear mixtures. Despite its significance, the identifiability of nonlinear ICA is known to be impossible without additional assumptions. Recent advances have proposed conditions on the connective structure from sources to observed variables, known as Structural Sparsity, to achieve identifiability in an unsupervised manner. However, the sparsity constraint may not hold universally for all sources in practice. Furthermore, the assumptions of bijectivity of the mixing process and independence among all sources, which arise from the setting of ICA, may also be violated in many real-world scenarios. To address these limitations and generalize nonlinear ICA, we propose a set of new identifiability results in the general settings of undercompleteness, partial sparsity and source dependence, and flexible grouping structures. Specifically, we prove identifiability when there are more observed variables than sources (undercomplete), and when certain sparsity and/or source independence assumptions are not met for some changing sources. Moreover, we show that even in cases with flexible grouping structures (e.g., part of the sources can be divided into irreducible independent groups with various sizes), appropriate identifiability results can also be established. Theoretical claims are supported empirically on both synthetic and real-world datasets.

📄 PDF Abstract BibTeX arXiv:2311.00866

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically
ICA _Independent component analysis (ICA) is a statistical and computational technique for revealing hidden factors that underlie sets of random variables, measurements, or…

Similar Papers 제목 키워드 기반

On the Identifiability of Nonlinear ICA: Sparsity and Beyond

2022-06-15 · Yujia Zheng, Ignavier Ng, Kun Zhang

Nonlinear independent component analysis (ICA) aims to recover the underlying independent latent sources from their observable nonlinear mixtures. How to make the nonlinear ICA model identifiable up to certain trivial in…

Inductive Bias

MC-GRU:a Multi-Channel GRU network for generalized nonlinear structural response prediction across structures

2025-03-10 · Shan He, Ruiyang Zhang

Accurate prediction of seismic responses and quantification of structural damage are critical in civil engineering. Traditional approaches such as finite element analysis could lack computational efficiency, especially f…

Computational Efficiency

On the Application of Data-Driven Deep Neural Networks in Linear and Nonlinear Structural Dynamics

2021-11-03 · Nan Feng, Guodong Zhang, Kapil Khandelwal

The use of deep neural network (DNN) models as surrogates for linear and nonlinear structural dynamical systems is explored. The goal is to develop DNN based surrogates to predict structural response, i.e., displacements…

Transfer Learning

Bridging Lottery Ticket and Grokking: Understanding Grokking from Inner Structure of Networks

2023-10-30 · Gouki Minegishi, Yusuke Iwasawa, Yutaka Matsuo

Grokking is an intriguing phenomenon of delayed generalization, where neural networks initially memorize training data with perfect accuracy but exhibit poor generalization, subsequently transitioning to a generalizing s…

Image ClassificationMemorization

Dynamic sparsity in tree-structured feed-forward layers at scale

2026-03-18 · Reza Sedghi, Robin Schiewer, Anand Subramoney, David Kappel arxiv

At typical context lengths, the feed-forward MLP block accounts for a large share of a transformer's compute budget, motivating sparse alternatives to dense MLP blocks. We study sparse, tree-structured feed-forward layer…

Question Answering