This thesis presents the Robot Imagination System (RIS). This system provides a convenient mechanism for a robot to learn a user's descriptive vocabulary, and how it relates to the world for action. With RIS, a user can describe unfamiliar objects to a robot, and the robot will understand the description as long as it is a combination of words that have been previously used to describe other objects.
One of the core uses of the RIS functionality is object recognition. Allowing requests with word combinations that have never been presented before together is well beyond the scope of many of the most relevant state of the art object recognition systems. RIS is not limited to object recognition. Through the use of evolutionary algorithms, the system endows the robot with the capability of generating a mental model (imagination) of a requested unfamiliar object. This capability allows the robot to work with this newly generated model within its simulations, or to expose the model to a user by projecting it on a screen or drawing the mental model as feedback so the user can provide a more detailed description if required.
A new paradigm for robot action based on consequences on the environment has been integrated within the RIS architecture. Changes in the environment are continuously tracked, and actions are considered complete when the performed effects are closest to the desired effects, in a closed perception loop. Experimental validations have been performed in real environments using the humanoid robot Teo, bringing the Robot Imagination System closer to everyday household environments in the near future.