The NMRDPP System

NMRDPP is controlled by a command language, which is read either from a file or interactively. The command language provides commands for the different phases (preprocessing, expansion, solution) of the methods, commands to inspect the resulting policy and value functions, e.g. with rendering via DOT [1], as well as supporting commands for timing and memory usage. A sample session, where the coin NMRDP is successively solved with PLTLSTR and PLTLMIN is shown in Figure 9. NMRDPP is implemented in C++, and makes use of a number of supporting libraries. In particular, it relies heavily on the CUDD package for manipulating ADDs [43]: action specification trees are converted into and stored as ADDs by the system, and moreover the structured algorithms rely heavily on CUDD for ADD computations. The state-based algorithms make use of the MTL - Matrix Template Library for matrix operations. MTL takes advantage of modern processor features such as MMX and SSE and provides efficient sparse matrix operations. We believe that our implementations of MDP solution methods are comparable with the state of the art. For instance, we found that our implementation of SPUDD is comparable in performance (within a factor of 2) to the reference implementation [29]. On the other hand, we believe that data structures used for regression and progression of temporal formulae could be optimised.