Previous research in machine learning has viewed the process of empirical discovery as search through a space of 'theoretical' terms. In this paper, we propose a problem space for empirical discovery, specifying six complementary operators for defining new terms that ease the statement of empirical laws. The six types of terms include: numeric attributes (such as PV/T); intrinsic properties (such as mass); composite objects (such as pairs of colliding balls); classes of objects (such as acids and alkalis); composite relations (such as chemical reactions); and classes of relations (such as combustion/oxidation). We review existing machine discovery systems in light of this framework, examining which parts of the problem space were, covered by these systems. Finally, we outline an integrated discovery system (IDS) we are constructing that includes all six of the operators and which should be able to discover a broad range of empirical laws.

One important component of learning is the ability to determine the correct conditions under which a rule should be applied. We review a number of systems that discover relevant conditions through a generalization process, and discuss some drawbacks of this approach. We then review an alternative approach to learning through discrimination, in which overly general rules are made more conservative when they lead to errors. Unlike generalization-based programs, a discrimination-based system is able to learn disjunctive rules, discover regularities in errorful data, recover from changes in the environment, and learn useful rules despite incomplete representations. We show how our theory of discrimination learning can be applied to the domains of concept attainment, strategy learning, first language acquisition, and cognitive development. Finally, we evaluate the theory along the dimensions of simplicity, generality, and fertility.