Against these theoretical and experimental objections to the Occam thesis there exists a body of apparent theoretical and empirical support.
Several attempts have been made to provide theoretical support for the Occam thesis in the machine learning context [Blumer, Ehrenfeucht, Haussler, and Warmuth, 1987, Pearl, 1978, Fayyad and Irani, 1990]. However, these proofs apply equally to any systematic learning bias that favors a small subset of the hypothesis space. Indeed, it has been argued that they equally support a preference for classifiers with high complexity [Schaffer, 1993, Berkman and Sandholm, 1995].
Holte [1993] compared learning very simple classification rules with the use of a sophisticated learner of complex decision trees. He found that, for a number of tasks from the UCI repository of machine learning datasets [ Murphy and Aha, 1993], the simple rules achieved accuracies of within a few percentage points of the complex trees. This could be considered as supportive of the Occam thesis. However, in no case did the simple rules outperform the more complex decision trees. Nor was it demonstrated that there did not exist yet another learning bias that consistently outperformed both those studied.
A final argument that might be considered to support the Occam thesis is that the majority of machine learning systems employ some form of Occam's razor and they appear to perform well in practice. However, it has not been demonstrated that even better performance would not be obtained if Occam's razor were abandoned.