SplineNets: Continuous Neural Decision Graphs
We present SplineNets, a practical and novel approach for using conditioning in convolutional neural networks (CNNs). SplineNets are continuous generalizations of neural decision graphs, and they can dramatically reduce runtime complexity and computation costs of CNNs, while maintaining or even increasing accuracy. Functions of SplineNets are both dynamic (i.e., conditioned on the input) and hierarchical (i.e., conditioned on the computational path). SplineNets employ a unified loss function with a desired level of smoothness over both the network and decision parameters, while allowing for sparse activation of a subset of nodes for individual samples. In particular, we embed infinitely many function weights (e.g. filters) on smooth, low dimensional manifolds parameterized by compact B-splines, which are indexed by a position parameter. Instead of sampling from a categorical distribution to pick a branch, samples choose a continuous position to pick a function weight. We further show that by maximizing the mutual information between spline positions and class labels, the network can be optimally utilized and specialized for classification tasks. Experiments show that our approach can significantly increase the accuracy of ResNets with negligible cost in speed, matching the precision of a 110 level ResNet with a 32 level SplineNet.
Code (0)
등록된 구현이 없습니다.
Tasks
PositionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Learning Efficient Surrogate Dynamic Models with Graph Spline Networks
While complex simulations of physical systems have been widely used in engineering and scientific computing, lowering their often prohibitive computational requirements has only recently been tackled by deep learning app…
Deep LearningFlowsheet synthesis through hierarchical reinforcement learning and graph neural networks
Process synthesis experiences a disruptive transformation accelerated by digitization and artificial intelligence. We propose a reinforcement learning algorithm for chemical process design based on a state-of-the-art act…
Chemical ProcessDecision MakingHierarchical Reinforcement Learningreinforcement-learning+2A Tutorial for Weighted Bipolar Argumentation with Continuous Dynamical Systems and the Java Library Attractor
Weighted bipolar argumentation frameworks allow modeling decision problems and online discussions by defining arguments and their relationships. The strength of arguments can be computed based on an initial weight and th…
Skill Discovery for Exploration and Planning using Deep Skill Graphs
Temporal abstraction provides an opportunity to drastically lower the decision making burden facing reinforcement learning agents in rich sensorimotor spaces. Well constructed hierarchies induce state and action abstrac…
continuous-controlContinuous ControlDecision MakingRight Place, Right Time! Dynamizing Topological Graphs for Embodied Navigation
Embodied Navigation tasks often involve constructing topological graphs of a scene during exploration to facilitate high-level planning and decision-making for execution in continuous environments. Prior literature makes…
Decision MakingLanguage ModelingLanguage ModellingLarge Language Model+2