Convex and Bilevel Optimization for Neuro-Symbolic Inference and Learning
We leverage convex and bilevel optimization techniques to develop a general gradient-based parameter learning framework for neural-symbolic (NeSy) systems. We demonstrate our framework with NeuPSL, a state-of-the-art NeSy architecture. To achieve this, we propose a smooth primal and dual formulation of NeuPSL inference and show learning gradients are functions of the optimal dual variables. Additionally, we develop a dual block coordinate descent algorithm for the new formulation that naturally exploits warm-starts. This leads to over 100x learning runtime improvements over the current best NeuPSL inference method. Finally, we provide extensive empirical evaluations across 8 datasets covering a range of tasks and demonstrate our learning framework achieves up to a 16% point prediction performance improvement over alternative learning methods.
Code (3)
Tasks
Bilevel OptimizationSimilar Papers 제목 키워드 기반
Learning Neuro-Symbolic Skills for Bilevel Planning
Decision-making is challenging in robotics environments with continuous object-centric states, continuous actions, long horizons, and sparse feedback. Hierarchical approaches, such as task and motion planning (TAMP), add…
Decision MakingMotion PlanningTask and Motion PlanningLearning Neuro-Symbolic Relational Transition Models for Bilevel Planning
In robotic domains, learning and planning are complicated by continuous state spaces, continuous action spaces, and long task horizons. In this work, we address these challenges with Neuro-Symbolic Relational Transition …
Model-based Reinforcement LearningBilevel Planning with Learned Symbolic Abstractions from Interaction Data
Intelligent agents must reason over both continuous dynamics and discrete representations to generate effective plans in complex environments. Previous studies have shown that symbolic abstractions can emerge from neural…
Efficiently Escaping Saddle Points in Bilevel Optimization
Bilevel optimization is one of the fundamental problems in machine learning and optimization. Recent theoretical developments in bilevel optimization focus on finding the first-order stationary points for nonconvex-stron…
Bilevel OptimizationAdaptive Mirror Descent Bilevel Optimization
In the paper, we propose a class of efficient adaptive bilevel methods based on mirror descent for nonconvex bilevel optimization, where its upper-level problem is nonconvex possibly with nonsmooth regularization, and it…
Bilevel Optimization