paper-with-me

Papers

A deep real options policy for sequential service region design and timing

2022-12-30 · Srushti Rath, Joseph Y. J. Chow

As various city agencies and mobility operators navigate toward innovative mobility solutions, there is a need for strategic flexibility in well-timed investment decisions in the design and timing of mobility service regions, i.e. cast as "real options" (RO). This problem becomes increasingly challenging with multiple interacting RO in such investments. We propose a scalable machine learning based RO framework for multi-period sequential service region design & timing problem for mobility-on-demand services, framed as a Markov decision process with non-stationary stochastic variables. A value function approximation policy from literature uses multi-option least squares Monte Carlo simulation to get a policy value for a set of interdependent investment decisions as deferral options (CR policy). The goal is to determine the optimal selection and timing of a set of zones to include in a service region. However, prior work required explicit enumeration of all possible sequences of investments. To address the combinatorial complexity of such enumeration, we propose a new variant "deep" RO policy using an efficient recurrent neural network (RNN) based ML method (CR-RNN policy) to sample sequences to forego the need for enumeration, making network design & timing policy tractable for large scale implementation. Experiments on multiple service region scenarios in New York City (NYC) shows the proposed policy substantially reduces the overall computational cost (time reduction for RO evaluation of > 90% of total investment sequences is achieved), with zero to near-zero gap compared to the benchmark. A case study of sequential service region design for expansion of MoD services in Brooklyn, NYC show that using the CR-RNN policy to determine optimal RO investment strategy yields a similar performance (0.5% within CR policy value) with significantly reduced computation time (about 5.4 times faster).

📄 PDF Abstract BibTeX arXiv:2212.14800

Code (0)

등록된 구현이 없습니다.

Tasks

Navigate

Methods 이 논문이 사용한 방법론

Golden Queue Managers 설명 없음

Similar Papers 제목 키워드 기반

Sequential Service Region Design with Capacity-Constrained Investment and Spillover Effect

2026-03-09 · Tingting Chen, Feng Chu, Jiantong Zhang arxiv

Service region design determines the geographic coverage of service networks, shaping long-term operational performance. Capital and operational constraints preclude simultaneous large-scale deployment, requiring expansi…

Using Options and Covariance Testing for Long Horizon Off-Policy Policy Evaluation

2017-03-09 · NeurIPS 2017 12 · Zhaohan Daniel Guo, Philip S. Thomas, Emma Brunskill

Evaluating a policy by deploying it in the real world can be risky and costly. Off-policy policy evaluation (OPE) algorithms use historical data collected from running a previous policy to evaluate a new policy, which pr…

Toward Discovering Options that Achieve Faster Planning

2022-05-25 · Yi Wan, Richard S. Sutton

We propose a new objective for option discovery that emphasizes the computational advantage of using options in planning. In a sequential machine, the speed of planning is proportional to the number of elementary operati…

SOAP-RL: Sequential Option Advantage Propagation for Reinforcement Learning in POMDP Environments

2024-07-26 · Shu Ishida, João F. Henriques

This work compares ways of extending Reinforcement Learning algorithms to Partially Observed Markov Decision Processes (POMDPs) with options. One view of options is as temporally extended action, which can be realized as…

MuJoCo

Semantics-based services for a low carbon society: An application on emissions trading system data and scenarios management

2015-02-09 · Cecilia Camporeale, Antonio De Nicola, Maria Luisa Villani

A low carbon society aims at fighting global warming by stimulating synergic efforts from governments, industry and scientific communities. Decision support systems should be adopted to provide policy makers with possibl…

Management