Multi-headed Neural Ensemble Search
Ensembles of CNN models trained with different seeds (also known as Deep Ensembles) are known to achieve superior performance over a single copy of the CNN. Neural Ensemble Search (NES) can further boost performance by adding architectural diversity. However, the scope of NES remains prohibitive under limited computational resources. In this work, we extend NES to multi-headed ensembles, which consist of a shared backbone attached to multiple prediction heads. Unlike Deep Ensembles, these multi-headed ensembles can be trained end to end, which enables us to leverage one-shot NAS methods to optimize an ensemble objective. With extensive empirical evaluations, we demonstrate that multi-headed ensemble search finds robust ensembles 3 times faster, while having comparable performance to other ensemble search methods, in both predictive performance and uncertainty calibration.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
EnsembleNet: End-to-End Optimization of Multi-headed Models
Ensembling is a universally useful approach to boost the performance of machine learning models. However, individual models in an ensemble were traditionally trained independently in separate stages without information a…
Hydra: Preserving Ensemble Diversity for Model Distillation
Ensembles of models have been empirically shown to improve predictive performance and to yield robust measures of uncertainty. However, they are expensive in computation and memory. Therefore, recent research has focused…
DiversitymodelHeaded-Span-Based Projective Dependency Parsing
We propose a new paradigm for projective dependency parsing based on headed spans. In a projective dependency tree, the subtree rooted at each word covers a contiguous sequence (i.e., a span) in the surface order. We c…
Dependency Parsingglobal-optimizationMulti-Target Radar Search and Track Using Sequence-Capable Deep Reinforcement Learning
The research addresses sensor task management for radar systems, focusing on efficiently searching and tracking multiple targets using reinforcement learning. The approach develops a 3D simulation environment with an act…
Deep Reinforcement LearningManagementreinforcement-learningReinforcement LearningHeaded-Span-Based Projective Dependency Parsing
We propose a new method for projective dependency parsing based on headed spans. In a projective dependency tree, the largest subtree rooted at each word covers a contiguous sequence (i.e., a span) in the surface order. …
Constituency ParsingDependency Parsing