Dependent Variables

We measure three aspects of the learning process: the resources consumed during learning, the characteristics of the resulting macro set and its utility:

• Learning resources: The resources consumed during the learning process until the learning system decides that it has learned enough. Note that the quiescence parameter significantly affects the learning resources. A higher value for this parameter increases resource consumption, but also decreases the likelihood that the problem solver will encounter a local minimum during testing.
  • Training problems: The total number of problems solved during training. The problem with this dependent variable is that it ignores the time invested in the search and the time invested in generating training problems.
  • Operator applications: Since the basic operation that is used both in search and in problem generation is the application of a basic operator to a state, we use the total number of operator applications as the principle measurement for the learning resources consumed.
• Macro-set statistics: Statistics about the characteristics of the generated macro set.
  • Total number: The total number of macros acquired during the learning session.
  • Mean Length: The average length of a macro.
  • Max length: The maximum length of a macro. This variable approximates the maximum radius.
• The utility of the acquired macros: According to Equation 2, the utility of the acquired macros depends on the cost of the search when using them. Therefore, our principle dependent variable should measure problem-solving speed.
  • CPU time: The most obvious candidate for measuring problem-solving speed is CPU time spent during search. However, such a measurement is overly affected by irrelevant factors such as hardware⁷, software and programming quality.
  • Expanded nodes: The number of nodes expanded during the search. This is a common method for measuring search speed. Nevertheless, this measurement may be misleading in the context of macro-learning, since the branching factor increases when acquiring macros.
  • Generated nodes: The number of nodes generated during search. This measurement takes into account the increased branching factor, but it does not account for the higher cost of generating a node by a macro due to the application of several basic operators.
  • Operator applications: The number of applications of a basic operator to a state. Note that we count every application, including those which are part of macro-operators and those which fail. This is the principle dependent variable, as it represents most accurately the problem-solving speed.
  • Solution quality: Macro-learning is not a suitable technique when the macros are used by an optimizing search procedure [20]. We are still interested, however, in the quality of the solution obtained. We measure the quality of the solution by its length in basic operators.