Papers Information Plane
“Information Plane” 태그가 달린 논문 30편 · 필터 해제
Information plane and compression-gnostic feedback in quantum machine learning
The information plane (Tishby et al. arXiv:physics/0004057, Shwartz-Ziv et al. arXiv:1703.00810) has been proposed as an analytical tool for studying the learning dynamics of neural networks. It provides quantitative ins…
Information PlaneQuantum Machine LearningEnhancing Neural Network Interpretability Through Conductance-Based Information Plane Analysis
The Information Plane is a conceptual framework used to analyze the flow of information in neural networks, but traditional methods based on activations may not fully capture the dynamics of information processing. This …
Information PlaneLightweight Conceptual Dictionary Learning for Text Classification Using Information Compression
We propose a novel, lightweight supervised dictionary learning framework for text classification based on data compression and representation. This two-phase algorithm initially employs the Lempel-Ziv-Welch (LZW) algorit…
Data CompressionDictionary LearningInformation Planetext-classification+1Cauchy-Schwarz Divergence Information Bottleneck for Regression
The information bottleneck (IB) approach is popular to improve the generalization, robustness and explainability of deep neural networks. Essentially, it aims to find a minimum sufficient representation $\mathbf{t}$ by s…
Adversarial RobustnessInformation PlaneregressionVariational InferenceInformation Plane Analysis Visualization in Deep Learning via Transfer Entropy
In a feedforward network, Transfer Entropy (TE) can be used to measure the influence that one layer has on another by quantifying the information transfer between them during training. According to the Information Bottle…
Deep LearningInformation PlaneEnd-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Training
End-to-end (E2E) training, optimizing the entire model through error backpropagation, fundamentally supports the advancements of deep learning. Despite its high performance, E2E training faces the problems of memory cons…
Information PlaneImproving the Robustness of Quantized Deep Neural Networks to White-Box Attacks using Stochastic Quantization and Information-Theoretic Ensemble Training
Most real-world applications that employ deep neural networks (DNNs) quantize them to low precision to reduce the compute needs. We present a method to improve the robustness of quantized DNNs to white-box adversarial at…
DiversityInformation PlaneQuantizationSoFaiR: Single Shot Fair Representation Learning
To avoid discriminatory uses of their data, organizations can learn to map them into a representation that filters out information related to sensitive attributes. However, all existing methods in fair representation lea…
FairnessInformation PlaneRepresentation LearningMutual information estimation for graph convolutional neural networks
Measuring model performance is a key issue for deep learning practitioners. However, we often lack the ability to explain why a specific architecture attains superior predictive accuracy for a given data set. Often, vali…
Inductive BiasInformation PlaneMutual Information EstimationHRel: Filter Pruning based on High Relevance between Activation Maps and Class Labels
This paper proposes an Information Bottleneck theory based filter pruning method that uses a statistical measure called Mutual Information (MI). The MI between filters and class labels, also called \textit{Relevance}, is…
Information PlaneA Comparative Genomic Analysis of Coronavirus Families Using Chaos Game Representation and Fisher-Shannon Complexity
From its first emergence in Wuhan, China in December, 2019 the COVID-19 pandemic has caused unprecedented health crisis throughout the world. The novel coronavirus disease is caused by severe acute respiratory syndrome c…
Information PlaneInformation flows of diverse autoencoders
The outstanding performance of deep learning in various fields has been a fundamental query, which can be potentially examined using information theory that interprets the learning process as the transmission and compres…
Information PlaneRepresentation LearningA Provably Convergent Information Bottleneck Solution via ADMM
The Information bottleneck (IB) method enables optimizing over the trade-off between compression of data and prediction accuracy of learned representations, and has successfully and robustly been applied to both supervis…
Information PlaneRepresentation LearningFundamental Limits and Tradeoffs in Invariant Representation Learning
A wide range of machine learning applications such as privacy-preserving learning, algorithmic fairness, and domain adaptation/generalization among others, involve learning invariant representations of the data that aim …
Domain AdaptationFairnessInformation PlanePrivacy Preserving+2Malicious Network Traffic Detection via Deep Learning: An Information Theoretic View
The attention that deep learning has garnered from the academic community and industry continues to grow year over year, and it has been said that we are in a new golden age of artificial intelligence research. However, …
BIG-bench Machine LearningFeature EngineeringInformation PlaneThe Dual Information Bottleneck
The Information Bottleneck (IB) framework is a general characterization of optimal representations obtained using a principled approach for balancing accuracy and complexity. Here we present a new framework, the Dual Inf…
Information PlaneOn the Information Plane of Autoencoders
The training dynamics of hidden layers in deep learning are poorly understood in theory. Recently, the Information Plane (IP) was proposed to analyze them, which is based on the information-theoretic concept of mutual in…
Information PlaneMutual Information EstimationOn Information Plane Analyses of Neural Network Classifiers -- A Review
We review the current literature concerned with information plane analyses of neural network classifiers. While the underlying information bottleneck theory and the claim that information-theoretic compression is causall…
Information PlaneMutual Information EstimationOn Predictive Information in RNNs
Certain biological neurons demonstrate a remarkable capability to optimally compress the history of sensory inputs while being maximally informative about the future. In this work, we investigate if the same can be said …
Information PlaneInformation Plane Analysis of Deep Neural Networks via Matrix-Based Renyi's Entropy and Tensor Kernels
Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained tremendous attention recently as a tool to gain insight into, among others, their generalization ability. However, it is by no means obvi…
Information Plane