Planner Assumption 2: What Are the Effect of Parameter Settings?

Performance of some planners does vary with the parameter settings. Unfortunately, it often is difficult to figure out how to set the parameters properly, and changing settings makes it difficult to compare results across experiments. Generally, this is not an issue because the developers and other users tend to rely on the default parameter settings. Unfortunately, sometimes the developers exploit alternative settings in their own experiments, complicating later comparison.

Recommendation 11: If a planner includes parameters, the developer should guide users in their settings. If they do not, then the default settings should be used by both the developers and others in experiments to facilitate comparison.