Reinforced Pipeline Optimization: Behaving Optimally with Non-Differentiabilities

2018-09-27 · Aijun Bai, Dongdong Chen, Gang Hua, Lu Yuan

Many machine learning systems are implemented as pipelines. A pipeline is essentially a chain/network of information processing units. As information flows in and out and gradients vice versa, ideally, a pipeline can be trained end-to-end via backpropagation provided with the right supervision and loss function. However, this is usually impossible in practice, because either the loss function itself may be non-differentiable, or there may exist some non-differentiable units. One popular way to superficially resolve this issue is to separate a pipeline into a set of differentiable sub-pipelines and train them with isolated loss functions. Yet, from a decision-theoretical point of view, this is equivalent to making myopic decisions using ad hoc heuristics along the pipeline while ignoring the real utility, which prevents the pipeline from behaving optimally. In this paper, we show that by converting a pipeline into a stochastic counterpart, it can then be trained end-to-end in the presence of non-differentiable parts. Thus, the resulting pipeline is optimal under certain conditions with respect to any criterion attached to it. In experiments, we apply the proposed approach - reinforced pipeline optimization - to Faster R-CNN, a state-of-the-art object detection pipeline, and obtain empirically near-optimal object detectors consistent with its base design in terms of mean average precision.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

object-detection Object Detection

Model-Based Reinforcement Learning via Meta-Policy Optimization

2018-09-14 · Ignasi Clavera, Jonas Rothfuss, John Schulman, Yasuhiro Fujita 외

Model-based reinforcement learning approaches carry the promise of being data efficient. However, due to challenges in learning dynamics models that sufficiently match the real-world dynamics, they struggle to achieve th…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Direct Mutation and Crossover in Genetic Algorithms Applied to Reinforcement Learning Tasks

2022-01-13 · Tarek Faycal, Claudio Zito

Neuroevolution has recently been shown to be quite competitive in reinforcement learning (RL) settings, and is able to alleviate some of the drawbacks of gradient-based approaches. This paper will focus on applying neuro…

OpenAI Gym reinforcement-learning Reinforcement Learning (RL)

Hyperspectral pan-sharpening: a variational convex constrained formulation to impose parallel level lines, solved with ADMM

2014-05-10 · Alexis Huck, François de Vieilleville, Pierre Weiss, Manuel Grizonnet

In this paper, we address the issue of hyperspectral pan-sharpening, which consists in fusing a (low spatial resolution) hyperspectral image HX and a (high spatial resolution) panchromatic image P to obtain a high spatia…

On Correctness of Automatic Differentiation for Non-Differentiable Functions

2020-06-12 · NeurIPS 2020 12 · Wonyeol Lee, Hangyeol Yu, Xavier Rival, Hongseok Yang

Differentiation lies at the core of many machine-learning algorithms, and is well-supported by popular autodiff systems, such as TensorFlow and PyTorch. Originally, these systems have been developed to compute derivative…

Online Robust Policy Learning in the Presence of Unknown Adversaries

2018-07-16 · NeurIPS 2018 12 · Aaron J. Havens, Zhanhong Jiang, Soumik Sarkar

The growing prospect of deep reinforcement learning (DRL) being used in cyber-physical systems has raised concerns around safety and robustness of autonomous agents. Recent work on generating adversarial attacks have sho…

Deep Reinforcement Learning OpenAI Gym reinforcement-learning Reinforcement Learning+1