Intelligent agents interacting with their environments combine information from several sense modalities and indulge in tasks that have components of perception, reasoning, learning and planning. Traditional AI systems focus on a single component. This paper highlights the importance of the integrated perceive-reason-act-learn loop, and describes a system designed to capture this loop. As a first step, it learns about simple objects, their qualities, and the words that name and describe them. The visual-linguistic associations formed serve as a bias in acquiring further knowledge about actions, which in turn aids the system in satisfying its internal needs (e.g., hunger, thirst, sleep, curiosity). Learning mechanisms that extract, aggregate, generate, de-generate and generalize build a hierarchical network (that serves as internal models of the environment) with which the system perceives and reasons.

The CHILDLIKE system is designed to learn about objects, object qualities, relationships among objects, and words that refer to them. Once sufficient visual-linguistic associations have been established, they can be used as foundations for a) further learning involving language alone and b) reasoning about the effect of different actions on perceived objects and relations, and internally sensed need levels. Here, we address the issue of learning efficient rules for action selection. A trial-and-error (or reinforcement) learning algorithm is used to acquire and refine action-related rules. Learning takes place via generation of hypotheses to guide movement through sequences of states, as well as modifications to two entities: the weight associated with each action, which encodes the uncertainty underlying the action, and the potential value (or vector) of each state which encodes the desirability of the state with respect to the current needs. CHILDLIKE is described, and issues relating to the handling of uncertainty, generalization of rules and the role of a short-term memory are also briefly addressed.