Papers Mutual Information Estimation
“Mutual Information Estimation” 태그가 달린 논문 91편 · 필터 해제
Closed-Form Information Capacity of Canonical Signaling Models
We employ a unified framework for computing the information capacity of biological signaling systems using Fisher Information. By deriving closed-form or easily computable information capacity formulas, we quantify how w…
DiversityFormMutual Information EstimationDistilling Transitional Pattern to Large Language Models for Multimodal Session-based Recommendation
Session-based recommendation (SBR) predicts the next item based on anonymous sessions. Traditional SBR explores user intents based on ID collaborations or auxiliary content. To further alleviate data sparsity and cold-st…
Mutual Information EstimationSession-Based RecommendationsA Deep Bayesian Nonparametric Framework for Robust Mutual Information Estimation
Mutual Information (MI) is a crucial measure for capturing dependencies between variables, but exact computation is challenging in high dimensions with intractable likelihoods, impacting accuracy and robustness. One idea…
Image GenerationMutual Information EstimationMeta-Statistical Learning: Supervised Learning of Statistical Inference
This work demonstrates that the tools and principles driving the success of large language models (LLMs) can be repurposed to tackle distribution-level tasks, where the goal is to predict properties of the data-generatin…
Mutual Information Estimationparameter estimationInfoBridge: Mutual Information estimation via Bridge Matching
Diffusion bridge models have recently become a powerful tool in the field of generative modeling. In this work, we leverage their power to address another important problem in machine learning and information theory, the…
Language ModelingLanguage ModellingMutual Information EstimationProtein Language ModelThe Decoupling Concept Bottleneck Model
The Concept Bottleneck Model (CBM) is an interpretable neural network that leverages high-level concepts to explainmodel decisions and conduct human-machine interaction. However, in real-world scenarios, the deficiency o…
modelMutual Information EstimationfastHDMI: Fast Mutual Information Estimation for High-Dimensional Data
In this paper, we introduce fastHDMI, a Python package designed for efficient variable screening in high-dimensional datasets, particularly neuroimaging data. This work pioneers the application of three mutual informatio…
Computational EfficiencyMutual Information EstimationVariable SelectionDisentangled Noisy Correspondence Learning
Cross-modal retrieval is crucial in understanding latent correspondences across modalities. However, existing methods implicitly assume well-matched training data, which is impractical as real-world data inevitably invol…
cross-modal alignmentCross-Modal RetrievalDisentanglementMutual Information EstimationArtifact Feature Purification for Cross-domain Detection of AI-generated Images
In the era of AIGC, the fast development of visual content generation technologies, such as diffusion models, bring potential security risks to our society. Existing generated image detection methods suffer from performa…
Mutual Information EstimationMutual Information Estimation via Normalizing Flows
We propose a novel approach to the problem of mutual information (MI) estimation via introducing a family of estimators based on normalizing flows. The estimator maps original data to the target distribution, for which M…
Mutual Information EstimationTowards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
Ensuring the trustworthiness of large language models (LLMs) is crucial. Most studies concentrate on fully pre-trained LLMs to better understand and improve LLMs' trustworthiness. In this paper, to reveal the untapped po…
FairnessMutual Information EstimationLarge Language Models are Efficient Learners of Noise-Robust Speech Recognition
Recent advances in large language models (LLMs) have promoted generative error correction (GER) for automatic speech recognition (ASR), which leverages the rich linguistic knowledge and powerful reasoning ability of LLMs…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)DenoisingKnowledge Distillation+5On the Properties and Estimation of Pointwise Mutual Information Profiles
The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables. One of its important properties is that its expected value is precise…
Mutual Information EstimationUncertainty QuantificationMINDE: Mutual Information Neural Diffusion Estimation
In this work we present a new method for the estimation of Mutual Information (MI) between random variables. Our approach is based on an original interpretation of the Girsanov theorem, which allows us to use score-based…
Mutual Information EstimationRethinking Negative Pairs in Code Search
Recently, contrastive learning has become a key component in fine-tuning code search models for software development efficiency and effectiveness. It pulls together positive code snippets while pushing negative samples a…
Code SearchContrastive LearningMutual Information EstimationRepresentation LearningOn regularized Radon-Nikodym differentiation
We discuss the problem of estimating Radon-Nikodym derivatives. This problem appears in various applications, such as covariate shift adaptation, likelihood-ratio testing, mutual information estimation, and conditional p…
Mutual Information EstimationZero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization
Zero-shot skeleton-based action recognition aims to recognize actions of unseen categories after training on data of seen categories. The key is to build the connection between visual and semantic space from seen to unse…
Action RecognitionMutual Information EstimationSkeleton Based Action RecognitionZero Shot Skeletal Action Recognition+1Subgraph Networks Based Contrastive Learning
Graph contrastive learning (GCL), as a self-supervised learning method, can solve the problem of annotated data scarcity. It mines explicit features in unannotated graphs to generate favorable graph representations for d…
AttributeContrastive LearningMutual Information EstimationSelf-Supervised Learning+1On the Effectiveness of Hybrid Mutual Information Estimation
Estimating the mutual information from samples from a joint distribution is a challenging problem in both science and engineering. In this work, we realize a variational bound that generalizes both discriminative and gen…
Mutual Information EstimationQuantizationMutual Information Estimation via $f$-Divergence and Data Derangements
Estimating mutual information accurately is pivotal across diverse applications, from machine learning to communications and biology, enabling us to gain insights into the inner mechanisms of complex systems. Yet, dealin…
Mutual Information Estimation