A New System

Our second contribution is NMRDPP, the first reported implementation of NMRDP solution methods. NMRDPP is designed as a software platform for their development and experimentation under a common interface. Given a description of the actions in a domain, NMRDPP lets the user play with and compare various encoding styles for non-Markovian rewards and search control knowledge, various translations of the resulting NMRDP into MDP, and various MDP solution methods. While solving the problem, it can be made to record a range of statistics about the space and time behaviour of the algorithms. It also supports the graphical display of the MDPs and policies generated. While NMRDPP's primary interest is in the treatment of non-Markovian rewards, it is also a competitive platform for decision-theoretic planning with purely Markovian rewards. In the First International Probabilistic Planning Competition, NMRDPP was able to enrol in both the domain-independent and hand-coded tracks, attempting all problems featuring in the contest. Thanks to its use of search control-knowledge, it scored a second place in the hand-coded track which featured probabilistic variants of blocks world and logistics problems. More surprisingly, it also scored second in the domain-independent subtrack consisting of all problems that were not taken from the blocks world and logistic domains. Most of these latter problems had not been released to the participants prior to the competition.