A non-stationary partially-observable markov decision process for planning in volatile environments

Abstract

In this paper, we consider the problem of timely optimizing and adapting the structure of public transit networks, taking into consideration the dynamic changes in average traffic congestion and average gross demand trends for specific transit routes, as well as the targets set by policy makers regarding maximum customer waiting times, and fuel economy and environmental preservation goals. To solve this problem, we sketch in this paper a method that allows for automatically determining the number of bus stops, and their optimal ordering and placement, based on ideas from the field of Machine Learning. Most importantly, our method allows for the continuous dynamic adaptation of the structure of the public transit system to echo any significant changes in customers behavior or the traffic landscape. To achieve our goals, we devise a novel Partially Observable Markov Decision Process (POMDP) model with dynamic nonparametric Bayesian nature, and devise efficient learning and planning algorithms based on this model.