Papers Generalization Bounds
“Generalization Bounds” 태그가 달린 논문 686편 · 필터 해제
Information-Theoretic Generalization Bounds of Replay-based Continual Learning
Continual learning (CL) has emerged as a dominant paradigm for acquiring knowledge from sequential tasks while avoiding catastrophic forgetting. Although many CL methods have been proposed to show impressive empirical pe…
Continual LearningGeneralization BoundsTuning without Peeking: Provable Privacy and Generalization Bounds for LLM Post-Training
Gradient-based optimization is the workhorse of deep learning, offering efficient and scalable training via backpropagation. However, its reliance on large volumes of labeled data raises privacy and security concerns suc…
Data PoisoningGeneralization BoundsTight Generalization Error Bounds for Stochastic Gradient Descent in Non-convex Learning
Stochastic Gradient Descent (SGD) is fundamental for training deep neural networks, especially in non-convex settings. Understanding SGD's generalization properties is crucial for ensuring robust model performance on uns…
Generalization BoundsSharp Generalization Bounds for Foundation Models with Asymmetric Randomized Low-Rank Adapters
Low-Rank Adaptation (LoRA) has emerged as a widely adopted parameter-efficient fine-tuning (PEFT) technique for foundation models. Recent work has highlighted an inherent asymmetry in the initialization of LoRA's low-ran…
Generalization Boundsparameter-efficient fine-tuningStability Analysis of Physics-Informed Neural Networks via Variational Coercivity, Perturbation Bounds, and Concentration Estimates
We develop a rigorous stability framework for Physics-Informed Neural Networks (PINNs) grounded in variational analysis, operator coercivity, and explicit perturbation theory. PINNs approximate solutions to partial diffe…
Generalization BoundsFairness Overfitting in Machine Learning: An Information-Theoretic Perspective
Despite substantial progress in promoting fairness in high-stake applications using machine learning models, existing methods often modify the training process, such as through regularizers or other interventions, but la…
FairnessGeneralization BoundsNeural Drift Estimation for Ergodic Diffusions: Non-parametric Analysis and Numerical Exploration
We take into consideration generalization bounds for the problem of the estimation of the drift component for ergodic stochastic differential equations, when the estimator is a ReLU neural network and the estimation is n…
Generalization BoundsEpistemic Errors of Imperfect Multitask Learners When Distributions Shift
When data are noisy, a statistical learner's goal is to resolve epistemic uncertainty about the data it will encounter at test-time, i.e., to identify the distribution of test (target) data. Many real-world learning sett…
AttributeGeneralization BoundsTransfer LearningLearning Where to Learn: Training Distribution Selection for Provable OOD Performance
Out-of-distribution (OOD) generalization remains a fundamental challenge in machine learning. Models trained on one data distribution often experience substantial performance degradation when evaluated on shifted or unse…
Bilevel OptimizationGeneralization BoundsOperator learningOperator Learning for Schrödinger Equation: Unitarity, Error Bounds, and Time Generalization
We consider the problem of learning the evolution operator for the time-dependent Schr\"{o}dinger equation, where the Hamiltonian may vary with time. Existing neural network-based surrogates often ignore fundamental prop…
Generalization BoundsOperator learningFast Rate Bounds for Multi-Task and Meta-Learning with Different Sample Sizes
We present new fast-rate generalization bounds for multi-task and meta-learning in the unbalanced setting, i.e. when the tasks have training sets of different sizes, as is typically the case in real-world scenarios. Prev…
Generalization BoundsMeta-LearningGeneralization Through Growth: Hidden Dynamics Controls Depth Dependence
Recent theory has reduced the depth dependence of generalization bounds from exponential to polynomial and even depth-independent rates, yet these results remain tied to specific architectures and Euclidean inputs. We pr…
Generalization BoundsGraph Representational Learning: When Does More Expressivity Hurt Generalization?
Graph Neural Networks (GNNs) are powerful tools for learning on structured data, yet the relationship between their expressivity and predictive performance remains unclear. We introduce a family of premetrics that captur…
Generalization BoundsEfficient End-to-End Learning for Decision-Making: A Meta-Optimization Approach
End-to-end learning has become a widely applicable and studied problem in training predictive ML models to be aware of their impact on downstream decision-making tasks. These end-to-end models often outperform traditiona…
Computational EfficiencyDecision MakingGeneralization BoundsStochastic OptimizationGeneralization Bounds for Quantum Learning via Rényi Divergences
This work advances the theoretical understanding of quantum learning by establishing a new family of upper bounds on the expected generalization error of quantum learning algorithms, leveraging the framework introduced b…
Generalization BoundsGeneralization Bounds and Stopping Rules for Learning with Self-Selected Data
Many learning paradigms self-select training data in light of previously learned parameters. Examples include active learning, semi-supervised learning, bandits, or boosting. Rodemann et al. (2024) unify them under the f…
Active LearningGeneralization BoundsvalidModel Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Laws
This paper formalizes an emerging learning paradigm that uses a trained model as a reference to guide and enhance the training of a target model through strategic data selection or weighting, named $\textbf{model steerin…
Contrastive LearningGeneralization BoundsmodelGeneralization Analysis for Contrastive Representation Learning under Non-IID Settings
Contrastive Representation Learning (CRL) has achieved impressive success in various domains in recent years. Nevertheless, the theoretical understanding of the generalization behavior of CRL is limited. Moreover, to the…
Contrastive LearningGeneralization BoundsRepresentation LearningGeneralization Guarantees for Multi-View Representation Learning and Application to Regularization via Gaussian Product Mixture Prior
We study the problem of distributed multi-view representation learning. In this problem, $K$ agents observe each one distinct, possibly statistically correlated, view and independently extracts from it a suitable represe…
DecoderGeneralization BoundsRepresentation LearningAchieving Distributive Justice in Federated Learning via Uncertainty Quantification
Client-level fairness metrics for federated learning are used to ensure that all clients in a federation either: a) have similar final performance on their local data distributions (i.e., client parity), or b) obtain fin…
EthicsFairnessFederated LearningGeneralization Bounds+1