It is believed that the consciousness of a sensory experience evolves from precise variations in contingencies between sensory and motor signals, and one of the overall goals of the SENSOPAC project is to understand the sensorimotor foundation of perception and cognition. Work within the project shall ultimately result in a machine that is able to discover the sensorimotor relationships by active sensing and exploratory actions, and that can generate a cognitive notion of objects through interaction. For this, a key issue is the ability to estimate and switch between contexts of the dynamics, which arise e.g. from manipulating different objects.

Such interaction with a changing environment causes non-stationarity of the dynamics and introduces significant complications in motor learning and control, since the learned dynamics model will have to be constantly adapted to the current situation. If contexts reoccur, then forgetting what has been previously learned and relearning it is a suboptimal strategy. The goal of this work is to overcome this problem and use the previously acquired experience to deal with re-occurring contexts.

Discrete contexts

The first approach is to assume that there is a finite number of contexts and formulate a multiple model scenario: instead of using and adapting only one dynamics model, a set of models is used, each of which is appropriate for a different environment. The first key issue is how to robustly determine the correct model for any given situation, which we refer to as context estimation. Robust context estimates are important both for control and learning of the models. We approach this problem by formulating a probabilistic model that represents the context as a latent switching variable. The context can then be estimated using simple probabilistic inference in an online fashion.

We also look into bootstrapping the different models from unlabeled data, which we refer to as context separation. Hereto, we utilize an EM-like algorithm to assign training data stemming from unknown situations to separate models and to adapt these models [1].

Continuous contexts

The discrete context, multiple model paradigm is limited insofar as the number of contexts has to be determined beforehand and that it is not immediately clear how to generalize between different contexts, an ability that humans seem to possess. We aim to overcome these problems by augmenting a single model with continuous hidden contextual variables.

Learning in this setting is a very difficult task and it is imperative to exploit any prior knowledge we might have about the relationship between the dynamics and appropriate contextual variables. As an example, the dynamics of a robot arm are linearly related to the inertial properties of the manipulator links. Manipulating different loads effectively affects the dynamics only by changing the inertial parameters of the last link of the manipulator. This relationship can be utilized to obtain the augmented model [2].

Incorporating sensor information

While the experienced dynamics may for many scenarios be sufficient for estimating the context, it is reasonable to assume that use of additional sensory information should enhance context estimation and learning under non-stationary conditions. In particular we look at incorporating measurements of tactile sensors to derive context estimates. Furthermore, we investigate an alternative approach to dealing with non-stationarity. In particular, we try to learn a single dynamics model that takes as additional input (sensory information: tactile or haptic data) that directly reflects the current context. This way, the process of (explicit) context estimation is not needed anymore [3].

SENSOPAC Deliverables

A pilot study on context switching has been handed in as Deliverable D2.2 in January 2008. We also made available some supplementary material.

Related Publications

[1]

Petkos, G., Toussaint, M. & Vijayakumar, S., Learning multiple models of nonlinear dynamics for control under varying contexts. Proc. International Conference on Artificial Neural Networks (ICANN). Athens, Greece (2006). [pdf]

[2]

Petkos, G., & Vijayakumar, S., Context estimation and learning control through latent variable extraction: From discrete to continuous contexts. Proc. IEEE International Conference on Robotics and Automation (ICRA). Rome, Italy (2007). [pdf]

[3]

Hoffmann, H., Petkos, G., Bitzer, S., & Vijayakumar, S., Sensor assisted adaptive motor control under continuously varying context. Proc. International Conference on Informatics in Control, Automation and Robotics (ICINCO). Angers, France (2007). [pdf]