Decision-Theoretic Planning with Asynchronous Events

Hakan L Younes

Abstract

I will introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, I will show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing MDP techniques. The techniques I will present can also be seen as an alternative approach for solving SMDPs, and I will demonstrate that the introduction of phases enables the generation of higher quality policies than those obtained by standard SMDP solution techniques. The talk is based on a paper that will be presented at AAAI-04. The paper is available on-line: http://www.cs.cmu.edu/~lorens/papers/aaai04.html

Back to the Main Page