The Information Sieve
We introduce a new framework for unsupervised learning of representations based on a novel hierarchical decomposition of information. Intuitively, data is passed through a series of progressively fine-grained sieves. Each layer of the sieve recovers a single latent factor that is maximally informative about multivariate dependence in the data. The data is transformed after each pass so that the remaining unexplained information trickles down to the next layer. Ultimately, we are left with a set of latent factors explaining all the dependence in the original data and remainder information consisting of independent noise. We present a practical implementation of this framework for discrete variables and apply it to a variety of fundamental tasks in unsupervised learning including independent component analysis, lossy and lossless compression, and predicting missing values in data.
Code (2)
Tasks
Missing ValuesSimilar Papers 제목 키워드 기반
A Multi-Pass Sieve Coreference Resolution for Indonesian
Coreference resolution is an NLP task to find out whether the set of referring expressions belong to the same concept in discourse. A multi-pass sieve is a deterministic coreference model that implements several layers o…
coreference-resolutionCoreference ResolutionDeepSieve: Information Sieving via LLM-as-a-Knowledge-Router
Large Language Models (LLMs) excel at many reasoning tasks but struggle with knowledge-intensive queries due to their inability to dynamically access up-to-date or domain-specific information. Retrieval-Augmented Generat…
AutoIE: An Automated Framework for Information Extraction from Scientific Literature
In the rapidly evolving field of scientific research, efficiently extracting key information from the burgeoning volume of scientific papers remains a formidable challenge. This paper introduces an innovative framework d…
Document Layout AnalysisManagementOopsieVerse: A Safety Benchmark with Damage-Aware Simulation for Robot Manipulation
While robotic manipulation capabilities have advanced rapidly, physical safety remains a major barrier to deploying household robots: task success is insufficient if the robot damages itself or its surroundings. Simulati…
Reinforcement LearningRobot ManipulationEvent Coreference Resolution with Multi-Pass Sieves
Multi-pass sieve approaches have been successfully applied to entity coreference resolution and many other tasks in natural language processing (NLP), owing in part to the ease of designing high-precision rules for these…
Avgcoreference-resolutionCoreference ResolutionEvent Coreference Resolution+1