Beyond the storage capacity: data driven satisfiability transition
Data structure has a dramatic impact on the properties of neural networks, yet its significance in the established theoretical frameworks is poorly understood. Here we compute the Vapnik-Chervonenkis entropy of a kernel machine operating on data grouped into equally labelled subsets. At variance with the unstructured scenario, entropy is non-monotonic in the size of the training set, and displays an additional critical point besides the storage capacity. Remarkably, the same behavior occurs in margin classifiers even with randomly labelled data, as is elucidated by identifying the synaptic volume encoding the transition. These findings reveal aspects of expressivity lying beyond the condensed description provided by the storage capacity, and they indicate the path towards more realistic bounds for the generalization error of neural networks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Capacity Credit Evaluation of Generalized Energy Storage Considering Strategic Capacity Withholding and Decision-Dependent Uncertainty
This paper proposes a novel capacity credit evaluation framework to accurately quantify the contribution of generalized energy storage (GES) to resource adequacy, considering both strategic capacity withholding and decis…
Decision MakingSchedulingPredicting Strategic Energy Storage Behaviors
Energy storage are strategic participants in electricity markets to arbitrage price differences. Future power system operators must understand and predict strategic storage arbitrage behaviors for market power monitoring…
Explainable AI: Deep Reinforcement Learning Agents for Residential Demand Side Cost Savings in Smart Grids
Motivated by recent advancements in Deep Reinforcement Learning (RL), we have developed an RL agent to manage the operation of storage devices in a household and is designed to maximize demand-side cost savings. The prop…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Universal Battery Degradation Forecasting Driven by Foundation Model Across Diverse Chemistries and Conditions
Accurate forecasting of battery capacity fade is essential for the safety, reliability, and long-term efficiency of energy storage systems. However, the strong heterogeneity across cell chemistries, form factors, and ope…
Representation LearningDistributed storage algorithms with optimal tradeoffs
One of the primary objectives of a distributed storage system is to reliably store large amounts of source data for long durations using a large number $N$ of unreliable storage nodes, each with $c$ bits of storage capac…