Norm-Based Capacity Control in Neural Networks
We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Understanding Weight Normalized Deep Neural Networks with Rectified Linear Units
This paper presents a general framework for norm-based capacity control for $L_{p,q}$ weight normalized deep neural networks. We establish the upper bound on the Rademacher complexities of this family. With an $L_{p,q}$ …
The Hidden Power of Normalization Layers in Neural Networks: Exponential Capacity Control
Normalization layers are critical components of modern AI systems, such as ChatGPT, Gemini, DeepSeek, etc. Empirically, they are known to stabilize training dynamics and improve generalization ability. However, the under…
Capacity Control of ReLU Neural Networks by Basis-path Norm
Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown t…
The Shape of Generalization through the Lens of Norm-based Capacity Control
Understanding how the test risk scales with model complexity is a central question in machine learning. Classical theory is challenged by the learning curves observed for large over-parametrized deep networks. Capacity m…
DiNADO: Norm-Disentangled Neurally-Decomposed Oracles for Controlling Language Models
NeurAlly-Decomposed Oracle (NADO) is a powerful approach for controllable generation with large language models. It is designed to avoid catastrophic forgetting while achieving guaranteed convergence to an entropy-maximi…
Machine Translation