Context Estimation and Switching|
Relates to SENSOPAC Work Package 2: 2A.1, 2A.3, 2C.1, 2C.2, 2C.4
People involved: Giorgos Petkos, Heiko Hoffmann, Sebastian Bitzer, Sethu Vijayakumar
Role within SENSOPAC
We investigate how a robot can possibly discriminate between objects through interaction. To this end, we develop algorithms for estimating and switching between contexts (e.g. different loads, moments of inertia, texture). Our work can be roughly divided into the three cases of
It is believed that the consciousness of a sensory experience evolves from precise variations in contingencies between sensory and motor signals, and one of the overall goals of the SENSOPAC project is to understand the sensorimotor foundation of perception and cognition. Work within the project shall ultimately result in a machine that is able to discover the sensorimotor relationships by active sensing and exploratory actions, and that can generate a cognitive notion of objects through interaction. For this, a key issue is the ability to estimate and switch between contexts of the dynamics, which arise e.g. from manipulating different objects.
Such interaction with a changing environment causes non-stationarity of the dynamics and introduces significant complications in motor learning and control, since the learned dynamics model will have to be constantly adapted to the current situation. If contexts reoccur, then forgetting what has been previously learned and relearning it is a suboptimal strategy. The goal of this work is to overcome this problem and use the previously acquired experience to deal with re-occurring contexts.
The first approach is to assume that there is a finite number of contexts and formulate a multiple model scenario: instead of using and adapting only one dynamics model, a set of models is used, each of which is appropriate for a different environment. The first key issue is how to robustly determine the correct model for any given situation, which we refer to as context estimation. Robust context estimates are important both for control and learning of the models. We approach this problem by formulating a probabilistic model that represents the context as a latent switching variable. The context can then be estimated using simple probabilistic inference in an online fashion.
We also look into bootstrapping the different models from unlabeled data, which we refer to as context separation. Hereto, we utilize an EM-like algorithm to assign training data stemming from unknown situations to separate models and to adapt these models .
The discrete context, multiple model paradigm is limited insofar as the number of contexts has to be determined beforehand and that it is not immediately clear how to generalize between different contexts, an ability that humans seem to possess. We aim to overcome these problems by augmenting a single model with continuous hidden contextual variables.
Learning in this setting is a very difficult task and it is imperative to exploit any prior knowledge we might have about the relationship between the dynamics and appropriate contextual variables. As an example, the dynamics of a robot arm are linearly related to the inertial properties of the manipulator links. Manipulating different loads effectively affects the dynamics only by changing the inertial parameters of the last link of the manipulator. This relationship can be utilized to obtain the augmented model .
Incorporating sensor information
While the experienced dynamics may for many scenarios be sufficient for estimating the context, it is reasonable to assume that use of additional sensory information should enhance context estimation and learning under non-stationary conditions. In particular we look at incorporating measurements of tactile sensors to derive context estimates. Furthermore, we investigate an alternative approach to dealing with non-stationarity. In particular, we try to learn a single dynamics model that takes as additional input (sensory information: tactile or haptic data) that directly reflects the current context. This way, the process of (explicit) context estimation is not needed anymore .
A pilot study on context switching has been handed in as Deliverable D2.2 in January 2008. We also made available some supplementary material.