PMAX stands for predicted maximum. It and all the other algorithms that folow, choose an experiment by assigning a value to each possible new experiment. Their suggestion is made by searching the space to find the one with the best value. Therefore, each of these algorithms are described by stating what their experiment valuation function is.
We call the vector of parameters, or inputs, X. We call the vector of results, or outputs, Y. We use C(X,Y) to represent the cost (or reward) for performing experiment X and obtaining result Y. Y = f(X) is the function mapping parameter settings onto results. E(f(X)) or E(Y|X) is what our function approximator tells us the expected result of a particular experiment is given all the data its been trained on so far. V(X) is the function that assigns a value to each experiment X. Using that notation PMAX is defined as:
In words, the value is just the cost of the experiment, given that its result is the expected result. PMAX will suggest the experiment which has the largest, or smallest, value depending on whether we are maximizing or minimizing (whether C is a cost or a reward).