A non-stationary infinite partially-observable markov decision process
Date Issued
2014
DOI
10.1007/978-3-319-11179-7_45
Abstract
Partially Observable Markov Decision Processes (POMDPs) have been met with great success in planning domains where agents must balance actions that provide knowledge and actions that provide reward. Recently, nonparametric Bayesian methods have been success-
fully applied to POMDPs to obviate the need of a priori knowledge of the size of the state space, allowing to assume that the number of visited
states may grow as the agent explores its environment. These approaches rely on the assumption that the agent's environment remains stationary; however, in real-world scenarios the environment may change over time. In this work, we aim to address this inadequacy by introducing a dynamic nonparametric Bayesian POMDP model that both allows for automatic inference of the (distributional) representations of POMDP states, and for capturing non-stationarity in the modeled environments. Formulation of our method is based on imposition of a suitable dynamic hierarchical Dirichlet process (dHDP) prior over state transitions. We derive e cientalgorithms for model inference and action planning and evaluate it on several benchmark tasks.
fully applied to POMDPs to obviate the need of a priori knowledge of the size of the state space, allowing to assume that the number of visited
states may grow as the agent explores its environment. These approaches rely on the assumption that the agent's environment remains stationary; however, in real-world scenarios the environment may change over time. In this work, we aim to address this inadequacy by introducing a dynamic nonparametric Bayesian POMDP model that both allows for automatic inference of the (distributional) representations of POMDP states, and for capturing non-stationarity in the modeled environments. Formulation of our method is based on imposition of a suitable dynamic hierarchical Dirichlet process (dHDP) prior over state transitions. We derive e cientalgorithms for model inference and action planning and evaluate it on several benchmark tasks.
File(s)![Thumbnail Image]()
Name
Chatzis.pdf
Size
196.62 KB
Format
Adobe PDF
Checksum (MD5)
b576e23b618afce35779a49a881da3a9