SHADE: Information Based Regularization for Deep Learning
Regularization is a big issue for training deep neural networks. In this paper, we propose a new information-theory-based regularization scheme named SHADE for SHAnnon DEcay. The originality of the approach is to define a prior based on conditional entropy, which explicitly decouples the learning of invariant representations in the regularizer and the learning of correlations between inputs and labels in the data fitting term. Our second contribution is to derive a stochastic version of the regularizer compatible with deep learning, resulting in a tractable training scheme. We empirically validate the efficiency of our approach to improve classification performances compared to common regularization schemes on several standard architectures.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningGeneral ClassificationSimilar Papers 제목 키워드 기반
SHADE: SHAnnon DEcay Information-Based Regularization for Deep Learning
Regularization is a big issue for training deep neural networks. In this paper, we propose a new information-theory-based regularization scheme named SHADE for SHAnnon DEcay. The originality of the approach is to define …
Deep LearningGeneral ClassificationSHADE: Information-Based Regularization for Deep Learning
Regularization is a big issue for training deep neural networks. In this paper, we propose a new information-theory-based regularization scheme named SHADE for SHAnnon DEcay. The originality of the approach is to define …
Deep LearningGeneral ClassificationDeepShade: Enable Shade Simulation by Text-conditioned Image Generation
Heatwaves pose a significant threat to public health, especially as global warming intensifies. However, current routing systems (e.g., online maps) fail to incorporate shade information due to the difficulty of estimati…
Contrastive LearningImage GenerationShadeBench: A Benchmark Dataset for Building Shade Simulation in Sustainable Society
Urban heat exposure is becoming an increasingly critical challenge due to the intensifying urban heat island effect. Fine-grained shade patterns, especially those induced by urban buildings, strongly influence pedestrian…
A Tool for the Procedural Generation of Shaders using Interactive Evolutionary Algorithms
We present a tool for exploring the design space of shaders using an interactive evolutionary algorithm integrated with the Unity editor, a well-known commercial tool for video game development. Our framework leverages t…
Evolutionary AlgorithmsUnity