Neural Extensions: Training Neural Networks with Set Functions
Integrating discrete computational steps into deep learning architectures is an important consideration when learning to reason over discrete items. However, many tasks that involve discrete choices are defined via (combinatorial) set functions, and thereby pose challenges for end-to-end training. In this work, we explore a general framework to construct continuous extensions of such discrete functions that enables training via gradient methods. Our framework includes well-known extensions such as the Lovasz extension of submodular set functions and facilitates the design of novel continuous extensions based on problem-specific considerations, including constraints. We demonstrate the versatility of our framework on tasks ranging from combinatorial optimization to image classification.
Code (0)
등록된 구현이 없습니다.
Tasks
Combinatorial Optimizationimage-classificationImage ClassificationSimilar Papers 제목 키워드 기반
Slack and Margin Rescaling as Convex Extensions of Supermodular Functions
Slack and margin rescaling are variants of the structured output SVM, which is frequently applied to problems in computer vision such as image segmentation, object localization, and learning parts based object models. Th…
Image SegmentationObject LocalizationSemantic SegmentationStructured PredictionNeural Set Function Extensions: Learning with Discrete Functions in High Dimensions
Integrating functions on discrete domains into neural networks is key to developing their capability to reason about discrete objects. But, discrete domains are (1) not naturally amenable to gradient-based optimization, …
Combinatorial OptimizationVocal Bursts Intensity PredictionActivations Through Extensions: A Framework To Boost Performance Of Neural Networks
Activation functions are non-linearities in neural networks that allow them to learn complex mapping between inputs and outputs. Typical choices for activation functions are ReLU, Tanh, Sigmoid etc., where the choice gen…
Time SeriesBayesian Optimization for Dynamic Problems
We propose practical extensions to Bayesian optimization for solving dynamic problems. We model dynamic objective functions using spatiotemporal Gaussian process priors which capture all the instances of the functions ov…
Bayesian OptimizationComplexity of Neural Network Training and ETR: Extensions with Effectively Continuous Functions
We study the complexity of the problem of training neural networks defined via various activation functions. The training problem is known to be existsR-complete with respect to linear activation functions and the ReLU a…