Benchmark Problems

The preponderance of problems in planning test sets are ``toy problems'': well-known synthetic problems designed to test some attribute of planners. The Blocksworld domain has long been included in any evaluation because it is well known, can have subgoal interactions and supports constructing increasingly complex problems (e.g., towers of more blocks). A few benchmark problems are simplified versions of realistic planning problems, e.g., the flat tire, refrigerator repair or logistics domains. We used the set included with the UCPOP planner. These problems were contributed by a large number of people and include multiple encodings of some problems/domains, especially Blocksworld.