With LOESS, the design parameters are the the size and shape of the kernel. As described earlier, we arbitrarily chose to work with a Gaussian kernel; we used the variance-based method for automatically selecting the kernel size.
Figure 6:
Variance and MSE learning curves for LOESS model trained on the Arm2D
domain. Dotted lines denote standard error for average of 60 runs,
each started with a single initial random example.
In the case of LOESS, both the variance and the MSE of the learner using the variance-minimizing criterion are significantly lower than those of the learner selecting data randomly. It is worth noting that on the Arm2D domain, this form of locally weighted regression also significantly outperforms both the mixture of Gaussians and the neural networks discussed by Cohn [1994].