Tom Larkworthy, Characterization of Self-Reconfiguring System State Spaces
Self-Reconfiguring Systems (SRS)(a.k.a. Transformers) offer various
advantages over fixed shaped robots, but to realize these advantages a
motion planning method is needed to control the self-reconfiguration
process. SRSs have the ability to be scaled to systems with the tens
of thousands of degrees of freedom, which far exceeds the capabilities
of traditional planning approaches. Yet SRS are typically thousands of
repeated modules on some uniform embedded space, symmetry is
everywhere, so surely there exists an exploitable sub-structure?
One efficient approach has been to plan in groups of meta-modules.
With suitable meta-module motion primitives designed by hand, the
reconfiguration state space for the coarse meta-modules is simpler
than the underlying model's cumbersome motion constraints. A drawback
is that resolution is lost by the definition of axis aligned
meta-modules. We have a better approach that adds just enough
constraints to hide the difficult motion primitives, leading to a near
linear algorithm for the hexagonal metamorphic robot. The planning
simplifies for a sub-space, but by sacrificing less.
Planning efficiently for a highly abstracted SRS like the 2D hexagonal
metamorphic robot is not particularly useful though. We need to
understand *why* our approach yields the observed benefits. We have
had some significant success de-mystifying the process of adding
constraints to an SRS motion model to create a new motion model that
is easier to plan with. In particular:
1. Let R = a raw SRS reconfiguration state space and C some other. C
is a further constrained version of R iff R < C where < denotes the
graph minor relation. The graph minor relation explains why some
meta-module state spaces can be run on some underlying model. It
concretely defines what is meant by sub-space in the SRS context.
2. Let C_n denote the reconfiguration state space for some model where
n denotes the number of units in the configuration. Motion models that
afford efficient planning solutions (e.g. meta-modularization and our
new algorithm) are well ordered by the graph minor relation i.e. C_n <
C_(n+1). Well-ordered states spaces by the graph minor relation
explain why some spaces can be tackled in a recursive, local and
iterative manner (all of which suggest the existence of some efficient
planning strategy)
3. Models that are efficient to plan with are highly connected and do
not contain bottlenecks (in a specific sense). High connectivity
explains why a greedy planning methodology suffices for certain
sub-spaces.
Thus we now understand when one state-space is a sub-space of another,
and when a particular sub-space affords efficient planning strategies.
These are broad rigorous principles that should inform SRS motion
planning algorithm design across different SRS architectures. The door
is now open to an SRS motion meta-planner.
Andreas Andreou, A micro-Doppler Sonar for Acoustic Scene Analysis
Fundamental to natural cognitive systems is the ability to detect and differentiate other living
creatures in the world and to characterize their behavior. The spatio-temporal patterns of the body and its
articulated components provide behavioral signatures and a means of communication among individuals
within the environment. Sound is the primary medium for long distance passive and active interaction
between animals and between animals and their environment; ranging from human speech communication
to the active auditory scene analysis of bats and dolphins using bio-sonar. A component of the research in
my lab is aimed bio-inspired autonomous acoustic scene analysis and decision making.
In this talk I will introduce briefly a software/hardware a distributed sensor networked system [1],
that is capable of forming composite representations of animate entities in the world exclusively through
the use of information derived from sounds. The system employs sound in two ways. Firstly, actively
through the emission, detection and processing of micro-Doppler sonar signals, the system is able to detect,
identify and classify moving articulated objects in the environment. Secondly, passively through the
processing and categorization of sounds emitted by the objects themselves, the system learns to recognize
the acoustic communications of living entities and to associate these messages with their detected behavior.
This situated cognitive system thus goes beyond human capabilities and is essentially an acoustic analogy
to a camera-based visual scene analysis system; one which is particularly suited to detecting the presence
and characterizing the behavior of living entities.
The bulk of the talk will focus on the micro-Doppler sonar system [2] for imaging moving
articulated objects in the environment. The device is inspired by natural bio-sonar which echo-locating
animals can use to locate, range and identify objects in their environment. Unlike much bio-sonar-inspired
robotics work, which has primarily focused on object identification and navigation, we use it to acquire
signatures that are employed in learning and classification of spectro-temporal patterns which characterize
explicit movements. These patterns are detected as modulations of the frequency of the emitted sonar
signal. Sonar technology complements cameras and visual surveillance in situations where the mere
presence of life is relevant (for security or search and rescue reasons, for example), since although it
depends upon a clear line 'of sight' between the detector and the object of interest it does not rely on
visibility per se.
The velocity of a moving object relative to an observer can be estimated by measuring the
frequency shift of a wave radiated or scattered by the object, known as the Doppler effect. If the object
itself contains moving parts, each moving part will result in a modulation of the base shift (the micro-
Doppler effect). For example, the frequency spectrum of acoustic or electromagnetic waves scattered from
a walking person is a complex time-frequency representation of human gait. It includes not only the
Doppler shifted components from the velocity of the entire body but also the micro-Doppler components
from the motion of the arms and legs. In the case of an articulated body such as a walking person, the torso,
each arm, and each leg has its own velocity, and even when the torso's velocity is constant, the velocity of
the limbs changes over time. The Doppler signature for such a complex object has multiple time-dependent
frequency shifted components corresponding to the velocity of the torso or an individual limb as a function
of time. A two-dimensional representation of human gait can be obtained from the returned Doppler signal
by applying a short-time Fourier transform (STFT) to the received signal
[1] D.H. Goldberg, A.G. Andreou, P. Julian, P.O. Pouliquen, L. Riddle and R. Rosasco,
Algorithm and VLSI implementation of a wake-up subsystem for an acoustic surveillance sensor
network, ACM Transactions on Sensor Networks, Vol. 2, No. 4, pp. 594-611, November 2006.
[2] Z. Zhang, P.O. Pouliquen, A. Waxman and A.G. Andreou, Acoustic micro-Doppler radar for
human gait imaging, Journal of the Acoustical Society of America Express Letters, Vol. 121, No. 3, pp.
110-113, March 2007.
Paulina Varshavskaya, Modeling Team Tactics for Sports Science and Robocup
I will present ongoing work in modeling and learning tactical play patterns in team sports such as football. The problem is to represent, and learn from video demonstrations, implicit coordination between team players in an adversarial situation. This representation should model the abstract, invariant essence of the play tactic, while discarding any irrelevant specifics such as exact player positions on the pitch. We have two goals in mind. On the one hand, it should enable us to automatically extract, store and compare patterns and instances of team decision-making, to be used in sports science. On the other hand, the model should be able to generate player behavior in the context of a game, to be used for high-level control in robot soccer. We treat this problem as hidden state estimation in a Dynamic Bayes Net. In this talk, I will go over the problem and relevant prior work in machine vision, behavior analysis and robotics. I will also present our first experiments in developing these models and very preliminary results.
This work is part of the IDEAlab project "Automating and Enhancing Team Sports Performance Analysis".
Jan-Peter Calliess, No-Regret Learning and a Mechanism for Distributed Multiagent Planning
In this talk I will outline the conceptual ideas of a novel mechanism for coordinated, distributed multiagent planning. We considered problems stated as a collection of single-agent planning problems coupled by common soft constraints on resource consumption. A key idea is to recast the distributed planning problem as learning in a repeated game between the original agents and a newly introduced group of adversarial agents who influence prices for the resources. The adversarial agents are set up to benefit from arbitrage: that is, their incentive is to uncover violations of the resource usage constraints and, by selfishly adjusting prices, encourage the original agents to avoid plans that cause such violations. If all agents employ no-regret learning algorithms in the course of this repeated interaction, we are able to show that our mechanism can be set up to achieve design goals such as social optimality and Nash-equilibrium convergence to within an error which approaches zero as the agents gain experience. In particular, the agents' average plans converge to a socially optimal solution for the original planning task. As an illustrating application, we consider a multiagent-based source routing task that can be successfully solved with our coordination mechanism.
Yijun Xiao, 3D shape acquisition of objects in high motion using a stereo vision sensor
High-speed 3D shape acquisition is a cutting-edge research with many potential applications. In this talk, I'll introduce an application in bat behaviour study in the context of the EU CHIROPING project. First I'll give an overview of the project and describe the work to be carried out in Edinburgh. Then I'll talk about the 3D sensor we employed and present an empirical study of performance evaluation of the sensor. Results from real bat shape data we collected in Denmark and Panama will be demonstrated and discussed.
Joanna Young, Olfactory associative learning and locomotion in the fruit fly
In this talk I will give an overview of two projects that I am currently working on in the laboratory using the fruit fly Drosophila. Firstly, I will show some recent results on olfactory associative learning and secondly I will describe a project investigating a brain region called the Central Complex, which is thought to be involved in the higher control of locomotion in the fly.
Matt Howard, Transferring Impedance Control Strategies via Apprenticeship Learning
In this talk, I will describe my recent research in the direction of designing biomimetic controllers for variable impedance actuators in the context of the EU STIFF project. Specifically, I will present an imitation learning approach, whereby the goal is to learn impedance modulation strategies from recordings of behaviour (for example, that of humans) and transfer these to a robotic plant with very different actuators and dynamics. In contrast to previous approaches, where impedance characteristics are directly imitated, the method we propose uses task performance as the metric of imitation, ensuring that the learnt controllers are directly optimised for the hardware of the imitator. As a key ingredient, apprenticeship learning is used to model the optimisation criteria underlying observed behaviour, in order to frame a correspondent optimal control problem for the imitator. Using local optimal feedback control techniques, we can then find an appropriate impedance modulation strategy under the imitator's dynamics. I'll present some recent experiments testing the performance of the approach for transferring behaviour between systems with antagonistic actuation (including a biologically realistic two- joint, six-muscle model of the human arm) to robotic systems with controllable active or passive impedance.
Lucia Ballerini, Appearance based skin cancer diagnosis
DERMOFIT is a Wellcome Foundation funded research project. One goal of
the project is develop a tool that will allow non-experts to diagnose
skin lesions by taking advantage of the ability of humans to make
visual matches even when they are not able to describe the lesions
(using words) in a consistent way. In this talk I'll present our work
within the DERMOFIT project.
I'll present recent results on the following studies:
1. A Query-by-Example Content-Based Image Retrieval System of
Non-Melanoma Skin Lesions
In this part I'll focus on colour and texture features that have been
extracted from skin lesions.
I'll also present an evolutionary algorithm for composite feature synthesis.
2. Fuzzy Description of Skin Lesions
In this part I'll talk about a system for describing skin lesion based
on a human perception model.
This has been developed by a MSc student.
Barbara Webb, What is associated with what in associative learning?
I will follow up an issue raised in my previous seminar (on non-elemental learning in insects) which is that despite a large number of behavioural experiments, neuroscientific investigations and computational models, there are a number of gaps and inconsistencies in accounts of associative learning which are hard to resolve. I'll illustrate with some supposedly 'simple' examples of learning in insects that we are trying to model. Depending on time, I will also use this issue to illustrate some of the general methodological pitfalls of 'agent' or 'animat' modelling (as discussed in my recent Adaptive Behaviour (vol 17 no 4) article "Animals vs. Animats").
Ricardo Gutierrez-Osuna, A system-wide model of the olfactory pathway for chemosensor arrays
In this talk, I will describe a computational model for chemical sensor arrays inspired by information processing in the olfactory system. First, I will present a model of sensory convergence that leads to spatial representations consistent with those observed in the olfactory bulb. Next, I will describe models of lateral inhibition in the olfactory bulb that provide concentration normalization and contrast enhancement of odor patterns. Finally, I will propose a model of bulb-cortex interactions that can be used to perform odor segmentation and background suppression. Our models are validated on experimental data from temperature-modulated metal-oxide sensor, optical microbead arrays, and infrared absorption spectroscopy. I will conclude with a brief discussion of our current work on active sensing with Partially Observable Markov Decision Processes.
back to seminarsGeorg Martius, Goal-Oriented Behavior from Self-Organizing Control in Autonomous Robots
Self-organization and the phenomenon of emergence play an essential role in living systems and form a challenge to artificial life systems. This is not only because systems become more lifelike, but also since self-organisation may help in reducing the design efforts in creating complex behavior systems. We consider agents under the closed sensorimotor coupling paradigm with a certain cognitive ability realized by an internal forward model. We show the self-organization of behavior with different robotic platforms. Using novel mechanisms of guided self-organization we can shape the emerging behaviors. Independently a set of behavioral primitives can be obtained with a modified competing expert schema. We show that this together with reinforcent learning can be used to perform goal oriented tasks.
back to seminarsSubramanian Ramamoorthy, Autonomous decision making in financial markets
In this talk I will introduce a research direction I am beginning to pursue, at the interface of machine learning and quantitative finance/economics, and describe preliminary results from one specific project in the area.back to seminars
Jan Wessnitzer, Buridan assay and models of olfactory learning
This talk will summarise ongoing work on two fly-related projects: (i) I will very briefly introduce our investigations into locomotion using the Buridan assay and then (ii) I'll talk about our current modelling attempts on olfactory learning and memory expression. Feedback, discussion and ideas will be welcomed.back to seminars
Taku Komura, Motion Editing and Retargeting based on Spatial Relationships
In this talk, I will present a new approach to edit / retarget motions by multiple characters in close contact. Conventional methods of motion editing / retargeting suffer from collisions and penetrations when applying them to motions of close contact such as grabbing, holding, wrestling and dancing. In our research, we tackle this problem by using spatial relationships between the body segments as constraints. I will show preliminary experimental results of successfully editing such close interactions. The approach is also applicable to motions of deformable objects such as cloth, and I will briefly discuss about the possible future directions.
back to seminarsHe Wang, A topological representation to wrap objects
Wrapping is a difficult task that requires control of cloths / wrapping papers that has high degrees of freedom. Controlling articulated characters / robots to conduct such tasks is difficult due to the lack of an appropriate representation of the status. In this talk, I will first explain about a new topological representation that express the status of the cloth and the object. Next, I will show the results of preliminary experiments in which the cloth is guided to wrap around the target object while avoiding other obstacles using the newly proposed representation. Finally, I will explain about the future direction of the research and how it can benefit the animation and manufacturing industries.
back to seminarsShin Yoshizawa, On surface ridges and their use for shape analysis
Surface ridges, curves on a surface along which the surface bends sharply, are powerful shape descriptors. They are widely used for shape matching, interrogation, and visualization purposes. Mathematically, the ridges can be defined in terms of extrema of the surface principal curvatures along their corresponding lines of curvature. A reliable detection of the ridges on surfaces approximated by polygonal meshes is a difficult computational task because it requires an accurate estimation of surface curvatures and curvature derivatives. In this talk, I will present a fast, robust, and faithful method for detecting the ridges on surfaces approximated by dense triangle meshes. The foundations of the method are two simple curvature and curvature derivative formulas overlooked in modern textbooks and a new observation about inversion-invariant surface properties. Applications of salient surface ridges to adaptive mesh simplification purposes will be considered. My talk will also include a brief review of geometry and image processing tools used for modern cell biology applications.
back to seminarsBob Fisher, Skin Cancer Diagnosis Using 3D Data
There has been much research on the automated diagnosis of melanoma, using colour and colour dermoscopy images. What has not been explored much is the benefit that 3D surface shape data has to offer, and whether the it is possible to apply the same methods to non-melanoma skin cancers. The two key problems that researchers have addressed is the segmentation of the lesion from against the background skin, and discrimination between moles and melanomas. The talk will report recent results on lesion segmentation and diagnosis using colour data supplemented with registered range data. We apply this to several types of skin cancer that have not previously had much image analysis research. I'll also say a little about some recent work on 3D shape extraction from the 500 fps bat observation project.
back to seminarsMichael Herrmann, Being and time: Cross-modal distortion of temporal perception
It is a long-standing question whether the perception of time relies on a centralized internal clock or is modulated by task-specific mechanisms. In experiments on a time perception task and a concurrently performed motor task with visual feedback, we found time to appear longer when visually observed movement was faster. The actual execution of the tracking motion did not contribute to this effect, but impaired discrimination performance by dual-task interference. This study demonstrates direct integration of temporal information from different modalities and provides causal support for the notion that time perception and continuous motor timing rely on separate mechanisms. The results are consistent with Bayesian integration of modality-specific temporal information into a centralized ''temporal hub``, which may be subject to attentional modulation.
back to seminarsDjordje Mitrovic, A Theory of Impedance Control based on Internal Model uncertainty
Efficient human motor control is characterised by an extensive use of joint
impedance modulation, which to a large extent is achieved by co-contracting
antagonistic muscle pairs in a way that is beneficial to the specific task.
While there is much experimental evidence available for the use of impedance
control in the CNS, no generally-valid computational model of impedance
control derived from first principles have been proposed so far.
In my talk I will present a computational model for impedance control, which
describes muscle coactivation in human arm reaching tasks as an emerging
mechanism from first principles of optimality. We hypothesise that, in
conjunction with an appropriate antagonistic arm and motor variability
model, impedance control emerges from an optimisation process that minimises
prediction uncertainties of the internal model. Our model is formalized
within the theory of stochastic Optimal Feedback Control (OFC). More
specifically I am using our recently introduced extension to the ILQG
algorithm, namely "ILQG with learned dynamics (ILQG-LD)".
Marc Deisenroth, Efficient Reinforcement Learning for Motor Control
In contrast to humans or animals, artificial learners often require more trials when learning motor control tasks solely based on experience. Efficient autonomous learners will reduce the amount of engineering required to solve control problems. By using probabilistic forward models, we can employ two key ingredients of biological learning systems to speed up artificial learning. We present a consistent and coherent Bayesian framework that allows for efficient autonomous experience-based learning. We demonstrate the success of our learning algorithm by applying it to challenging nonlinear control problems in simulation and in hardware. For these tasks, we report an unprecedented speed of learning.
back to seminarsJinah Park, Visualization, Simulation and Interaction
Computer graphics deals with the computational generation of images and image sequences from given data stored in a virtual world, and visualization addresses the issues of casting data to suitable representations. Furthermore, computer haptics allow the users to feel by touch of the virtual objects. In this talk, I will overview the basic concepts of disciplines that transfer the data in virtual world to something that human can perceive, and introduce the related on-going research work at the Computer Graphics and Visualization Research Laboratory at KAIST.
back to seminarsNaoufel Werghi, Extracting topologically ordered features from 3D mesh surface: Framework, applications, and open issues
The triangular mesh is the most widely used representation for encoding surface shapes. However, it has the intrinsic drawback of lacking an ordered structure that would allow 3D systematic analysis and modeling. In the literature, this problem has been addressed using a planar or spherical parameterization of the triangle mesh surface. However, these solutions seem more suitable for shape modeling rather than shape analysis. In this talk, we present another alternative that consists in extracting ordered and structured topological features from the 3D triangular mesh. After exposing the approach, we highlight potential applications which include: assessing the tessellation quality of the triangular mesh, extraction of facial landmarks, computation of iso-geodesic rings, facial surface registration, computation of iso-geodesic rings, and definition of 3D facial local and global signatures.
back to seminarsSebastian Bitzer, Dimensionality Reduction for Reinforcement Learning
Policy gradient based reinforcement learning is a promising way of reinforcement learning in continuous spaces. For hard problems with large state and action spaces, however, it is still questionable whether policy gradient methods find a sufficient solution in acceptable time. Suitable initialisations of the policy with demonstrated successful or close to successful episodes have been used to overcome this problem by incorporating prior knowledge into the reinforcement learning problem. We are looking into a way of incorporating prior knowledge about the problem at hand when it is available as independent samples from the relevant part of the state space rather than as a complete solution. In this talk I will present the difficulties we are facing with policy gradient based reinforcement learning on the example of a toy problem and give an idea about how we intend to address some of them. Feedback highly encouraged!
back to seminarsJoern Diedrichsen, How the motor systems exploits redundancy
Redundancy is one of the defining features of biological motor systems: there are often more limbs, joints, and muscles available than strictly needed for a given task. In this talk I will show how the human motor system exploits this redundancy. First, I will address the principles that govern how work is distributed across different effectors. Secondly, I will show examples from bimanual movements of how the nervous system optimally changes the response to perturbations to exploit task-redundant dimensions. Finally, I argue that to understand learning in redundant systems, we need to consider two learning mechanisms: Error-based learning reacts to a perturbation of a movement by counteracting it on the next movements. Additionally to this prevalent response, there exists a second, opposing reaction: The nervous system also changes the movement plan in the direction of the perturbed movements through an associative learning mechanism. Along task-redundant dimensions these two mechanisms can be revealed due to their different time-courses. Together these two learning mechanisms dictate how redundant motor tasks are learned, sometimes resulting in stable, non-optimal solutions.
back to seminarsThomas Larkworthy, Accuracy prediction of redundant/parallel modular robots & Comparison of general methods for Self-reconfiguration planning
Self-reconfiguring systems (SRSs) offer various advantageous including
versatility, robustness etc. To be accepted in an industrial domain,
important parameters such as manipulation accuracy need to be
estimated. This talk will present experimental evidence that accuracy
can be increased trivially using redundant actuation. In addition,
attempts at modelling the accuracy of the experimental data are
presented.
--PLUS---
Planning how to coordinate the subcomponents of a SRS in order to
achieve a desired configuration from a starting configuration is a
hard problem. I present a comparison between popular general motion
planning algorithms, namely greedy search (GS), RRTConnect,
probabalistic roadmaps (PRM) and simulated annealing (SA). Two new
heuristics are introduced and compared against the classic optimal
assignment heuristic normally used in this domain. Finally, I compare
two solution smoothing algorithms.
In future work these two topics shall be fused to provide an algorithm
that will plan a transition from an arbitrary configuration of a SRS
into a configuration that meets user specified accuracy manipulation
criteria.
Matthew Whitaker, Reinforcement Learning in Multi-Robot Systems - The Role of Communication
Single-agent reinforcement learning algorithms rely on specific properties of Markov Decision Processes (MDPs) - namely that the environment modelled by the MDP has an observable state and the MDP is not dynamic. These properties are much weaker in Partially Observable MDPs (POMDPs), and cease to exist in POMDPs containing multiple adaptive agents (Decentralised/Distributed POMDPs).
Many multi-agent reinforcement learning algorithms attempt to address this problem by introducing new methods for calculating optimal policies, some of which specifically acknowledge the existence of other agents, and some that don't. Few of these algorithms consider the explicit use of communicative actions to share information between learning agents, and even less view this problem from a mobile robotics perspective.
In the first part of this talk I will review the ways in which MDPs have been expanded to accommodate the properties of multi-agent systems, and identify the dominant trends in algorithm design for this domain. I will then go on to discuss the failings of these algorithms when applied specifically to cooperative multi-robot scenarios where communication is not cost-free.
Alan McKinnon, A Noise-Bound Method for Detecting Shadow-Free Scene Changes in Image Sequences
To deal with the sensor noise always present in a captured image, many image processing applications use global filters and thresholds. We use a camera noise model to predict the noise in an image and use this information to identify shadow-free scene changes in image sequences, based on a dual-illumination algorithm which provides a colour-based method for shadow detection and removal. This talk will briefly describe the camera noise model and show how we have used it in the shadow-free scene change algorithm.
back to seminarsDanica Kragic, Active Vision for Detecting, Fixating, Manipulating Objects and Learning of Human Actions
The ability to autonomously acquire new knowledge through interaction
with the environment is one of the major research goals in the field
of robotics. The knowledge can be acquired only if suitable
perception-action capabilities are present. In other words, a robotic system has to be able to detect, attend to and manipulate objects in the environment.
In the first part of the talk, we present the results of our longterm work in the area of vision based sensing and control. The work on finding, attending, recognizing and manipulating objects in domestic environments is discussed. More precisely, we present a stereo based vision system framework where aspects of Top-down and Bottom-up attention and foveated attention are put into focus and demonstrate how the system can be utilized for object grasping.
The second part of the talk presents our work on the visual analysis of human manipulation actions which are of interest
for e.g. human-robot interaction applications where a robot learns
how to perform a task by watching a human. A method
for classifying manipulation actions in the context of the objects manipulated, and classifying objects in the context of the actions used to manipulate them is presented. The action-object correlation over time is then modeled using conditional random fields. Experimental comparison shows improvement in classification rate when the action-object correlation is taken into account, compared to separate classification of manipulation actions and manipulated objects.
Alexander Belyaev, Skeleton-based free-form shape deformations
In this talk, I present a skeleton-based mesh deformation approach. The approach is based on the classical medial axis transform and utilizes the use of the so-called differential coordinates. A multi-resolution version of the approach is also considered. In addition, interesting links between the proposed free-form shape deformation scheme and classical and modern results in the differential geometry of sphere congruencies (envelopes) are discussed. The talk is based on a joint work with Shin Yoshizawa (RIKEN) and Hans-Peter Seidel (MPI-Informatik).
back to seminarsHubert Shum, Simulating Interaction among Virtual Characters
In this seminar, I will briefly go through my works in the past three
years of my PhD study about simulating interactions among virtual
characters. I will also outline a new research project to model crowd
interactions using particle system / fluid dynamics for real-time
applications such as computer games.
Character animation has recently attracted more and more attention due
to the high demand in the game and the movie industries. While dense
interactions is a popular topic in both industries (considering the
huge amount of fighting scenes and action games), past researches
focus mainly on simulating the motion of a single character, or
simulating crowd movements where there is no interaction between
characters. The problem to model how multiple characters densely
interact is unsolved. In my studies, I apply artificial intelligence
techniques such that character interactions can be synthesized in a
realistic, fast, and controllable way. On one hand, I can generate
high quality interactions among characters such as fighting and
dancing. On the other hand, I can plan the movements of hundreds of
characters in real-time.
Matthew Howard, Learning Control Policies from Variable Constraint Data
An intuitive way to describe many everyday human behaviours is in terms of performing some task subject to dynamic constraints. Examples include opening a door, wiping a window and stirring soup in a saucepan, all of which involve interaction with the environment in a way that constrains the actions of a controller. In this talk I will present my work in learning from demonstration from data containing variable motion constraints, with a view to transferring behaviour to robots in a way that generalises over constraints. I will describe the novel methods we have developed to do this and recent experimental work for transferring human demonstrations to the ASIMO and DLR robots.
back to seminarsShu Lim Ho, Synthesizing human interactions with topological constraints
In computer graphics, many researchers have been working on synthesizing realistic human
animations. With the use of MOCAP systems, creating animation is no longer a
time-consuming task. However, most of the existing work focus on scenes without close
interactions of multiple avatars. In addition, it is difficult to capture such motions
due to the limitations of the MOCAP devices.
Using individually captured motions to generate multi-character animations with close
interactions is a practical approach. But it is expected that there will be a lot of
collisions among the body segments of the characters. Existing methods can handle these
problems by using collision detection algorithms and editing the positions of the
colliding segments by inverse kinematics. However, these methods do not take into account
the topological relationships between the body segments. Therefore, there is no guarantee
that the topological relationships will be kept the same after the editing.
In this talk, I will present our previous work on synthesizing close human interactions,
such as wrestling, in topology space. The new method can synthesize the transition
motions between different topological states. This method is not limited to synthesizing
multiple-character animations but also the interactions between character and objects
with topological constraints.
Heba Lakany, Brain Computer Interfaces
During the past decade, a growing interest has developed in recording, detecting and analysing brain (neural) signals to investigate, explore and understand human motor control systems with the aim of attempting to build interfaces that use real-time signals to generate commands to control and/or communicate with the environment. Such interfaces, referred to as Brain computer interfaces, could provide people with severe motor disabilities means for communication and control of augmentative and assistive devices.
In this talk, I shall briefly explain principles of brain computer interfaces and introduce recent research developments in this field. I will finally describe the research we carry out in the Bioengineering Unit at Strathclyde in this area and present some of our recent results.
Etienne Burdet, Motor learning: In humans, for robots
This talk will first present some of our activity at the interface of robotics and biology; it will then focus on the learning of force and impedance in humans. To succeed in motor tasks, in particular when using tools, humans need to control force and impedance in order to interact with the environment and its uncertainty in an appropriate way. Over the years, we have investigated this learning and have discovered a simple model of human motor adaptation, which correctly predicts the gradual change of muscle activation trial after trial. This adaptive algorithm can be used to develop adapted rehabilitation protocols, and gives rise to flexible robot behaviors.
back to seminarsFinlay Stewart, Modelling visual-olfactory integration in free-flying fruit flies
The ability of flying fruit flies (Drosophila melanogaster) to locate a hidden food odour source depends upon their visual surroundings - specifically, the environment must contain vertical contrasts. In this talk, I present data from my free-flight behavioural experiments investigating this phenomenon. I shall then discuss my attempts to model this behaviour in simulation. Visuomotor control is achieved by three parallel subsystems (collision avoidance, optomotor response, and speed regulation) which detect wide-field optic flow patterns. An odour plume is simulated based upon empirical recordings, and ways in which olfactory input can modulate the visuomotor controller are investigated. I find that having the olfactory signal modulate just two parameters of the model is sufficient to reproduce the behavioural effects seen in flies.
back to seminarsTakamitsu Matsubara, Learning full-body movement skills on humanoid robot via reinforcement learning and human motion prediction with multiple Gaussian process dynamical models
This talk consists of two parts.
In the first part, I would like to present results of our work about learning full-body movement skills on a humanoid robot
via Reinforcement Learning (RL). When we observe specific human movements, in each case the number of independent variables can be much reduced compared to a human's structural degrees of freedom. It suggests that more abstract features represent each human movement rather than joint coordinate, and its dimension could be extremely low. With this in mind, we propose a paradigm addressing motor learning on a humanoid robot via RL. In order to compose such a task-relevant low-dimensional feature space, appropriate coordination among all-joints is first introduced for the task. Then, RL is applied in a associated low-dimensional feature space to learn a policy for achieving a desired full-body movement skill. The effectiveness of the paradigm is validated through an application to learning Central Pattern Generator based biped walking and whole-body impact movement skill based on Inverted Pendulum Model.
In the second part, I would like to talk about preliminary results of our recent exploration of human motion modeling and prediction with Gaussian Process Dynamics Models (GPDMs). With the fact that each human motion has a low-dimensional intrinsic space, each human motion is modeled by a GPDM. Then, a human motion predictor is comprised of multiple GPDMs with a simple gating criterion. Its effective is demonstrated by a experiment with mo cap data.
Michael (Yijun) Xiao, Nonlinear image resizing based on seam carving
Alias is inevitable when resizing a digital image according to Nyquist-Shannon sampling theorem. In this talk, I?ll show two forms of alias, i.e., geometric distortion and intensity(colour) smoothness, in three widely-adopted image resizing methods, namely, nearest neighbour mapping, bilinear and bicubic interpolations. I?ll then present a nonlinear sampling scheme that can alter the proportion of geometric distortion and intensity smoothness in the alias of the resized images. I?ll give an example of applying such nonlinear scheme to resize digital images in fractional scale based on a technique called ?seam carving?. Preliminary results show that the proposed method distinctively outperforms some of the traditional image resizing methods. Open questions and ongoing research will be discussed at the end of talk.
back to seminarsTaku Komura, Computer animation in topology space
In this talk, I will first present the updates of our research about motion synthesis of character interactions as well as its future direction. I will first talk about the topology-based motion synthesis approach. By controlling the characters in the topology space, we can produce motions of a humanoid such as tangling its arms to the others, passing the hand through a hole and carrying bulky objects, by local optimization techniques. I will then present my ideas about its extension to handle 2D surfaces and 3D volumetric data, which will help to help create animating scenes that involve cloths, 3D objects, and objects composed of multiple layers. Next, I will present about Interaction Patches, which is an efficient method to compose scenes of multiple characters closely interacting with one another, such as one character fighting with others, a crowd falling down onto one another like dominos, a American football player running to the goal while avoiding the tackling defenders and a large number of characters passing luggage to one another. I will also explain about the on-going research in this direction.
back to seminarsStefan Klanke, Dimensionality reduction for movement data
I will talk about a few ideas on dimensionality reduction for movement data that I've been playing with every now and then. In particular, I will present the basics of Unsupervised Kernel Regression and an extension that facilitates handling periodic movements, along with some results from joint work with Jan Steffen (Bielefeld). I will also introduce a much simplified model for creating movement manifolds from data sequences. All this is very much work in progress, with more questions open than answered.
back to seminarsPaulina Varshvskaya, Reinforcement Learning in Distributed Robotic Systems
Many complex natural phenomena and engineered systems, from swarming locusts and fish to transportation networks, require distributed decision-making under uncertainty. For artificial distributed systems, including teams of robots and modular robots, decision-making controllers can be difficult for human engineers to design. Instead, we can make the individual elements of these systems automatically learn good policies. However, the distributed nature of the system and of the process results in major challenges for learning algorithms, including partial observability and action interference. In this talk, we focus on the problem of learning locomotion gaits for self-reconfiguring modular robots in order to explore and address the challenges of learning in distributed systems. We use a class of reinforcement learning algorithms based on policy search. We then explore ways to systematically bias search, and propose a novel algorithm based on local agreement, as a principled means to efficient distributed reinforcement learning in partially observable environments. Our experiments in simulation demonstrate fully distributed learning of good policies for locomotion and target-reaching tasks in lattice-based self-reconfiguring modular robots.
back to seminarsMatthijs Snel, PhD proposal: The Drive for Emergent Adaptive Behaviour: Embodied Neural Controllers in Dynamic Environments
Similar to the "constructivist" approach to intelligence, my research proposal follows the belief that intelligent behaviour and an understanding of it should be achieved by starting out from a minimal set of basic principles, and having behaviour emerge from the interactions between these principles. More concretely, the proposed research adopts as basic principles concepts that are also employed by nature to create intelligent agents: evolution, development, and learning, and internal drives such as hunger and thirst. The first three processes are used to shape neural controllers consisting of a motivation and behaviour module, based on internal drives. The controllers develop in an embodied context where simulated robots are subject to foraging tasks in different kinds of physically realistic environments; agent survival depends on the upkeep of its drives which in turn have to be satisfied by foraging and avoiding obstacles. The main hypotheses are that, firstly, drives can implicitly encode multiple agent goals and decouple them from environmental properties, thereby allowing an evolved, drive-based agent to outperform a pre-designed one under changing environmental conditions. Secondly, evolution, development and learning each present different adaptive advantages, and evolution and development can provide learning with a more effective bias than human designers can. In my talk I will outline how I intend to investigate these hypotheses, will present some preliminary results of this investigation, and will briefly touch on related work.
back to seminarsBalazs Csanad Csaji, On Parameter Uncertainties of Markov Decision Processes.
Sequential decision making under the presence of uncertainties is often modelled by Markov decision processes (MDPs). In practical applications, however, the parameters of MDPs are often not exactly known, they are usually estimated, and therefore, there may be parameter uncertainties, as well. In the presentation, first, I consider the case when the transition-probability function is fixed, but uncertain, viz., we do not know what it is, we only know that it belongs to a given uncertainty set. I propose an efficient robust convex optimization based approach to handle this problem. Then, I analyze the case, when the transition-probability and the immediate-cost functions are uncertain in a way that they may vary from time to time, provided that the accumulated changes remain asymptotically bounded. I investigate the possibility of applying stochastic iterative algorithms (SIAs) in this kind of changing environments. I present a generalized convergence theorem for SIAs with time-dependent update operator. Afterwards, I apply this theorem to deduce a convergence theorem for value function based reinforcement learning (RL) methods working in changing MDPs. Finally, I illustrate these results through variants of classical RL algorithms as well as numerical experiments.
back to seminarsBarbara Webb, Non-elemental learning: what is it, and can insects do it?
Insects are not just simple reactive creatures, but how "intelligent" are they? One way to investigate this issue is to look at what they are capable of learning. In particular we can consider certain associative learning paradigms that have been taken as evidence for more complex processing in mammals, for example, paradigms that suggest there is internal prediction of reward, or that involve recognition of compound stimuli or context. I will try to give a systematic outline of the relevant paradigms and the current status of evidence for whether insects can or cannot learn them.
back to seminarsSethu Vijayakumar, Bayesian Multisensory Perception and Sensorimotor Adaptation.
I will talk about the role of Bayesian structure inference in causality modelling and introduce domains where this framework is being increasingly used. Then I will focus on two paradigms. First, I consider multisensory cue integration, where we have applied this framework for realtime multimodal audio-visual tracking. Secondly, I will talk about a sensorimotor adaptation paradigm that predicts and elicits a surprising recalibration "after-effect" during force field adaptation based on ideal observer Bayesian modelling. This is work done with Timothy Hospedales and Adrian Haith.
back to seminarsIoannis Havoutis, PhD proposal: Learning robust humanoid behaviors using composable skill manifolds.
Motion synthesis for humanoid robot behaviours is made difficult by the combination of task space, joint space and kinodynamic constraints defining realisability and imprecision in available models of dynamics and these constraints. The traditional (sampling based) motion planning approach to this problem involves significant computational complexity and often requires ad hoc heuristics to handle constraints. Part of the inefficiency of such algorithms derives from the fact that they do not directly leverage the structure of the problem. On the other hand, machine learning methods that are able to learn this structure in a data-driven fashion are restricted to limited domains where data, e.g., from demonstration, is available. We present a novel framework for motion planning and control that leverages the complementary strengths of these approaches. We learn low-dimensional representations of skills in the form of manifolds corresponding to the possible configurations of a robot when performing a specific tasks. Then, in order to acquire sufficient information about all possible variations of the underlying task, we bias a randomized exploration process to refine the structure of these manifolds by actively acquiring additional data regarding task variations. A large scale behaviour involves composition of many such skill manifolds so that motion synthesis corresponds to a process of trajectory generation over an appropriate sequence of these manifolds. Once exploration has yielded sufficient data, the desired trajectory may be computed as the solution of a constrained optimisation problem. In this proposal, we demonstrate preliminary results supporting this idea of combining machine learning with sampling based motion planning and optimal control for a simulated humanoid robot performing a bipedal locomotion task. Our eventual goal is to devise a complete motion planing and control framework suitable for robust global control of full-body behaviours in humanoid robots. We outline the essential components of our proposed solution and briefly discuss their relation to existing ideas in robotics and avatar animation.
back to seminarsIan Saunders, PhD proposal: The role of sensory feedback for a closed-loop prosthetic hand.
The state-of-the-art in distal upper-extremity prostheses is an underactuated hand with few degrees of control. Consequently, physiological acceptance and hand dexterity are compromised. We hypothesise that a fundamentally limiting aspect of the amputee's learning process is the lack of sensory feedback. In an idealised experimental setup we will address this hypothesis under a variety of hand control methods and artificial sensory feedback methods, regarding the healthy human hand as a gold standard. Treating the resulting prosthesis as a model of the healthy hand, we we will attempt to answer some interesting open questions in human sensorimotor control. We will use the closed-loop device as a novel manipulandum which, unlike in healthy individuals, can be independently deprived of proprioception, exteroception and control at a fine temporal granularity.
back to seminarsHannes Saal, PhD proposal: Human and artificial tactile processing and sensorimotor control.
Imagine searching your coat pocket for your car keys. Your fingers get hold of a certain object and thousands of touch receptors in your fingertips start sending information to your brain about various object properties: how hard or soft it is, its texture, and properties of shape like curvature. Based on this feedback, you might disregard an object or explore it further if your expectations of how your keys should feel like match the incoming data. How is the sensory information processed so rapidly and how are suitable actions determined based on the sensory feedback? In my thesis I want to explore this problem both from a neuroscientific and a robotic point of view.
In this talk I will review my work on neural coding in the human tactile system, and present future plans involving haptic object recognition in a robotic system.
Paolo Favaro,Superresolution with the light field camera and 3D reconstruction with coded aperture photography
In this presentation we will present two methods that we have recently developed in the realm of computational photography: superresolution with the light field camera and shape from coded aperture. Light field cameras have been recently shown to be very effective in performing applications such as digital refocusing and 3D reconstruction. However, so far the proposed methods provide reconstructions that are limited by the number of microlenses, and as a result the final resolution is much lower than that of traditional imaging devices. We will present analysis and algorithms to overcome such limitations that exploit the redundancy of information present in light field images. In the second contribution we investigate the reconstruction of 3D scenes from a single image obtained from a coded aperture. We show how imposing regularization in the spatial domain during the reconstruction via graph cuts yields depth maps with resolutions comparable to those obtained from two images as in traditional stereo. Both algorithms are tested on synthetic and real data.
back to seminarsFranck Multon,Computer Simulation for retrieving plausible bipedal locomotion for living and fossilized hominids
Simulation is now widely used in biomechanics to investigate human motion control. Classical methods are based on applying knowledge on either kinematic or dynamic models, such as providing the average shape of the angular trajectories while walking. Hence, these methods are based on prerequired measurements performed on living subjects. In paleo-anthropology the problem is different because most of the subjects are fossils. Hence, retrieving a plausible gait for such kind of species is really challenging, as no measurements are available. Most of the approaches consequently compare the shape of bones of various existing species in order to identify the relationship between shape and function. However, these comparative approaches generally focus on some parts of the skeleton while the locomotor system is obviously not limited to one specific joint. Evolutionary robotics was also used to simulate gaits on Australopithecus affarensis (Lucy) by tuning muscle activation patterns. Despite the interesting results, the method was based on important assumptions on muscle attachments and was limited to 2D. We have proposed a new approach that consists in separating the degrees of freedom (DOF) into to parts: the movement of the feet and the joints responsible for moving the legs. The latter DOFs are retrieved by applying an inverse kinematics framework based on global hypotheses on motion control (such as minimizing energy, satisfying kinematic constraints with the ground and taking a rest posture into account). Retrieving a plausible trajectory of the feet is performed by optimization. It consists in optimizing the trajectories of the feet to make the creature walk into predefined footprints while minimizing energy and Jerk. The two layers are connected in an optimization loop which calculates the whole-body motion. This approach has been validated on humans and chimpanzees and has been applied to Australopithecus affarensis (Lucy).
back to seminarsFlexible, general purpose robots need to tailor their visual processing to their task on the fly. I'll describe a new approach to this within a planning framework, where the goal is to plan a sequence of visual operators to apply to the regions of interest (ROIs) in a scene. We pose the visual processing problem as a partially observable Markov decision problem (POMDP). This requires probabilistic models of operator effects to capture the quantitative unreliability of the processing actions, and thus reason precisely about trade-offs between plan execution time and plan reliability. Since planning in practical sized POMDPs is intractable we show how to ameliorate this intractability somewhat for our domain by defining a hierarchical POMDP. We compare both flat and hierarchical POMDP approaches with a continual planning approach. We show empirically on a real robot visual domain that all the planning methods outperform naive application of all visual operators. The key result is that the POMDP methods produce more robust plans than either naive visual processing or the CP approach. I'll argue that visual processing problems represent a challenging and worthwhile domain for planning techniques, and that our hierarchical POMDP based approach to them opens up a promising line of new research.
back to seminarsProf Florentin Worgotter,On the relation between reinforcement and hebbian learning
Hebbian learning could be seen as "the mother of all learning rules" in the brain as it can operate directly on individual synapses relying only on the mechanisms of LTP and LTD. To remain compatible with the biophysics of neurons, other, more complex rules like TD-learning or rules for supervised learning normally require a network implementation. Implementations at single synapses are often not possible. Different from this commonly observed situation, here we will show that it is indeed possible to directly emulate TD learning in a mathematically equivalent way by three-factor differential hebbian learning at single synapses. First we will discuss three-factor learning as such, showing some robot applications, then we will construct a system that emulates TD and finally we will discuss possible biophysical implications of this.
back to seminarsMichael Herrmann,Emergence of Agency in Adaptive Agents
Robotic agents can self-organize their interaction with the environment by an adaptive controller that simultaneously maximizes sensitivity of the behaviour and predictability of sensory inputs. Building on previous work with single robots, we study here the interaction of two such agents and show that quasi-social interactions among artificial agents may emerge in this so-called homeokinetic framework. The agents prefer actual encounters with an adaptive partner to an identical but replayed stimulus pattern, which might be interpreted as a realization of the concept of "agency".
back to seminarsProf R.B. Fisher,Skin cancer, Time Lapse Videos and Video-Ground Truthing
This talk will cover several topics that I have been involved with over the past year. Three short segments will follow with some question time. - The development of some skin cancer diagnosis techniques with Steven McDonagh - The production of enhanced time-lapse videos, with Jorge Reyes-Ortiz - The evaluation of video ground truthing, involving several people, particularly Thor List
back to seminarsJo-Anne Ting,Towards autonomous Bayesian real-time learning
I propose a set of Bayesian methods to help us work towards the goal of autonomous real-time learning. Specifically, I am interested in scenarios where the input data has thousands of dimensions and where real-time, incremental learning may be needed, as in robotics, real-time vision, brain-computer interfaces, autonomous vehicles etc. Real-time autonomous learning in such data-rich environments is challenging, due to issues such as outliers, noisy sensory data, redundant and irrelevant dimensions, and the need for computational efficiency in real-time conditions. I introduce a set of automatic methods to address these challenges, using Bayesian inference -- combined with variational approximations -- in order to eliminate open parameters in a principled way. All these methods can be leveraged together to develop a Bayesian local kernel shaping for nonlinear regression. Bayesian local kernel shaping is computationally efficient, requires no sampling and automatically rejects outliers. It can be used for nonparametric regression with local polynomials (e.g., for real-time learning) or as a novel method to achieve nonstationary regression with Gaussian processes. The usefulness and improved performance of our algorithms are illustrated in various robotic applications such as parameter identification in robot dynamics, real-time outlier detection in tracking and learning a task-level control law.
back to seminarsHannes Saal,Spatiotemporal distribution of tactile information across the human fingertip
IThe tactile system of the human fingertips provides information which is crucial for dexterous object manipulation and it does so quickly and reliably. Decoding of relevant sensory information from the neural responses of tactile primary afferents involves identifying features like curvature of objects in contact and direction of fingertip forces.
An important question is which neural codes could transmit rich tactile information rapidly and whether they are used in the tactile system. Another question is how the spatial arrangement of the tactile receptors on the fingertip as well as the mechanical properties of the skin influence the neural response. Taken together, we ask for a spatiotemporal characterization of the tactile information.
In this talk I will discuss how information theory can be applied to tackle these questions and present some results of my analysis of neural data from the human tactile system.
back to seminarsMatt Howard,Direct Policy Learning from Constrained Motion Data
Recent advances in the study of controllers for redundant manipulators have resulted in a generic constraint-based framework both for dynamic and kinematics-based control. An important part of this framework is control in the nullspace of constraints. Commonly, this is done using some control policy either hand-designed by the robot engineer or assumed given in schemes for learning control.
n my talk I will discuss two novel methods for determining nullspace policies by estimating them from movement data. In the first, given constrained movement trajectories, we make local models of a potential function and perform alignment of these to create a globally consistent policy model. This method makes a rather strong assumption that the policy is conservative (potential based), but has the advantage of being very data-efficient. In the second we introduce a novel risk functional that allows us to make a meaningful comparison between the estimated policy and constrained observations. This allows us to model any arbitrary policy directly from constrained observations using several regression techniques trained with the modified error function.
I will present results demonstrating the methods on systems of varying complexity including kinematic data from the ASIMO humanoid robot.
back to seminarsMatt Snel,Evolution of Valence Systems in an Unstable Environment
In artificial systems employing reinforcement learning (RL), reward was traditionally implemented by having a pre-designed reward function provide the agent with a scalar reinforcement signal. It is, however, debatable whether an agent that has to rely on a pre-designed reward function to learn can truly be called adaptive and autonomous. Several approaches for moving RL away from pre-designed reward functions exist; for example, Evolutionary RL (ERL), that inspired the approach that I will present in this talk, and, more recently, Intrinsically Motivated RL (IMRL).
As in ERL, we evolve agents using an actor-critic learning scheme based on homeostasis of internal drives like hunger and thirst. In particular, our paper compares the performance of drive- versus perception-based motivational systems in an unstable environment. We investigated the hypothesis that valence systems (systems that evaluate positive and negative nature of events) that are based on internal physiology will have an advantage over systems that are based purely on external sensory input.
Results show that inclusion of internal drive levels in valence system input significantly improves performance. Furthermore, a valence system based purely on internal drives outperforms a system that is additionally based on perceptual input. I provide arguments for why this is so and relate our architecture to brain areas involved in animal learning.
back to seminarsJan Steffen,Dextrous Grasping and Manipulation Using Manifolds
In dextrous hand control, the implementation of manipulation movements still is a complex and intricate undertaking. Often, a lot of object physics and modelling effort has to be incorporated into a controller working only for a restricted task specification and performing quite artificially looking movements.
In this talk, starting from a representation for dextrous grasping - the Grasp Manifold - I motivate and present adaptations which enable a modified representation - the Manipulation Manifold - to robustly represent manipulation movements. We use manifolds of hand postures embedded in the finger joint angle space which are constructed such that manipulation parameters including the advance in time are represented by distinct manifold dimensions. This allows for simple purposive navigation within such manifolds. I present the first steps towards the construction of such manifolds using the Unsupervised Kernel Regression (UKR) and the way of applying it for manipulation in the example of turning a bottle cap in a physics-based simulation.
Finally, I will give a short overview of the ideas and problems we would like to address during my research stay in Edinburgh in order to realise a more unsupervised construction of the presented movement manifolds.
back to seminarsDjordje Mitrovic,Adaptive Optimal Feedback Control for high-dimensional Movement Systems
In recent years Optimal Feedback Control (OFC) has become the predominant motion generation strategy for biological movement systems. OFC not only explains most motion patterns observed in human reaching but also allows a mathematically coherent formulation. This makes it an appealing theory for modeling volitional motion of biological and artificial systems.
Most OFC models make simplifying assumptions (linear dynamics model and quadratic cost function) due to the computational limitations OFC methods impose. In this proposal I investigate OFC for highly nonlinear and redundant systems, based on iterative optimal control methods. So far these methods relied on an analytic form of the system dynamics, which may often be unknown, difficult to estimate for more realistic control systems or may be subject to frequent systematic changes. I present a novel combination of learning a forward dynamics model within the OFC framework. Utilising such adaptive internal models can compensate for complex dynamic perturbations of the controlled system in an online fashion. This allows us, for the first time, to study OFC from an adaptation perspective and its link to biological motor control.
In my talk I will introduce this adaptive OFC framework and present results from several adaptation experiments in simulation. I further will motivate my planned future research towards a hardware implementation and a biological interpretation of my adaptive OFC model.
back to seminarsGiorgos Petkos,Learning dynamics for robot control under varying contexts
High performance control algorithms require an accurate model of the system's dynamics. Learning dynamics is an attractive alternative when accurate analytical derivation is not possible. However, dynamics may exhibit nonstationarity due to interaction with different environments or objects. We refer to the unobserved factors that affect the dynamics as the context of the dynamics. Under nonstationary dynamics, the dynamics model needs to be adapted whenever the context of the dynamics changes, otherwise there will be large tracking errors. In this talk, we examine ways to improve performance by reusing knowledge obtained by experiencing the dynamics of a set of contexts.
We consider two classes of scenarios. In the first the dynamics may switch between a finite set of contexts. In that case we use a set of learned models for each of the contexts and switch between them accordingly. We formulate this as a probabilistic discrete latent variable model. In a more complicated scenario with continuous, possibly infinite number of potential contexts, the use of a set of models may not be viable and generalization to novel contexts is required. To tackle this problem, we reformulate the probabilistic discrete latent variable model as a continuous latent variable model.
back to seminarsMark Payne,Mushroom Bodies and Motor Control
The mushroom bodies of the insect brain are highly prominent, highly structured regions which have been researched for many years. In common with parts of the mammalian brain which share these characteristics (e.g. the hippocampus, the cortex and the cerebellar cortex), the functional role of the mushroom bodies is not clearly understood. Compelling evidence has shown that they are involved in olfactory conditioning, but ablation studies and genetic mutants also point to a role in suppression and termination of locomotory bouts.
My own work into sensory integration in the cricket has focussed on the role of efference copies and sensory predictions, in particular the suppression of turning due to self-generated optical flow. In this talk I will discuss the components that are necessary in a system that performs such a task, how these might map onto the mushroom bodies, and present my attempts to implement the system in a spiking neural network for controlling a robot with sound-tracking and optomotor behaviours.
back to seminarsMichael Mangan,Navigation in insects using multiple place memories
In this talk I shall provide a brief overview of my current behavioural study in which I seek the mechanism underpinning idiosyncratic route formation in the desert ant Cataglyphis ibericus. I shall also present results from a previous behavioural study in which the ability of the cricket Gryllus bimaculatus to return to a hidden target location using only visual cues was tested.
I shall then compare the performance of different classes of homing algorithm in the various experimental paradigms outlined above. Furthermore, I shall discuss the limitations of such models and offer a novel solution using a classical neural network architecture. Preliminary results using this technique shall be presented and future work with the aim of forming a general model for route navigation in insects shall be discussed.
back to seminarsFlavio Prieto (National University of Columbia),Inspection of 3D Deformable Parts Using Radial Basis Functions
In industry, one of the most common schemes to perform automated inspection tasks consists of matching a design represented by a 3D CAD model with 3D data measured by geometric sensors such as laser scanner or industrial CT. To carry out this comparison it is necessary to first align or register both the CAD model with the part 3D measurements. Usually, the part to be inspected can be considered to be rigid where a simple rigid body transformation is enough to achieve alignment with the CAD model. However, modern manufactured parts are becoming more and more flexible due to new materials such as composites. Traditional inspection system requires in this case that the part have to be fixed in place using clamps. This process is time-consuming and difficult to automate. What is necessary is an inspection system capable of applying to the CAD model a more general transformation that includes the deformation of the parts to the traditional rigid body assumption. In this work, we explore the use of Radial Basis Functions (RBF): Gaussian, multiquadrics and inverse multiquadrics, as a method to represent the deformations, required for the models registration during the inspection processes. In order to do this, a comparison between the deformation obtained with the Radial Basis Functions and the one obtained using Finite Elements analysis is presented allowing us to calibrate the accuracy of the inspection process.
back to seminarsTom Larkworthy,Self-Reconfiguration Planning Heuristics
Planning algorithms for metamorphic systems based on search require heuristics because of the massive state space. The most popular heuristic has been the optimal assignment heuristic. This is an application of the well known combinatorial problem of optimal assignment. It can be calulated in O(n3).
I will present a suboptimal dynamic persistent version of the heuristic which can be calculated incrementally a cost of around O(n log(n)) (I have not worked it out yet, but its defiantly below n2). Although it arrives at sub optimal solutions to the optimal assignment problem, which translates during search to more states being evaluated during planning, the reduced computational cost more than makes up for this.
I will also present a very different form of heuristic which compresses a reconfiguration state into a fixed size vector representation. This can be calculated incrementally in linear time. The new heuristic performs worse than the suboptimal assignment heuristic when used in a greedy search. However, the fixed size vector representation allows more sophisticated meta-heuristic searches to be applied effeciently. Early results show that in some situations rapidly exploring random tree based planning using this new heuristic can outperform greedy search.
back to seminarsJafreezal Jaafar,Reactive Behaviour In Game Design Using Fuzzy Logic
Computer games are programs that enable a player to interact with a virtual game environment for entertainment and fun. Each game has its own strategy, action, curiosity, action, challenge and fantasy that make each game unique and interesting, which can essentially motivate games players. In this presentation, the classic game Pacman was chosen to demonstrate a behaviour based system using fuzzy logic. In earlier implementation of the game, the ghost logic did not realistically adjust to user skill and movement. For instance, the ghosts did not move toward the areas where the Pacman needed to complete the level. While this could be done with classic logic, fuzzy logic can provides a better way for a system to deal with the often ambiguous data required to implement such behaviours. In addition, this type of system allows rules to be easily added to increase the opponents? intelligence further. For these reasons, fuzzy logic has been chosen as the basis for the intelligent control of the ghosts? behaviour. I will present the design and implementation of a real-time fuzzy-based system for an interactive game. The chosen game is a remake of Pac-Man in which the opponents are BDI-style intelligent agents. The components of the system and the methods used in fuzzifying the game?s rules and variables are also discussed. Finally, will discuss the results observed in the implementation of the game along with a comparison to classical design methodologies.
back to seminarsTheodoros Damoulas (University of Glasgow),From Automated Currency Validation to Protein Fold Recognition: Probabilistic Multi-Class Multi-kernel Learning
In diverse machine learning problems ranging from automated currency validation (ACV) to protein fold prediction, we encounter the situation where multiple object descriptors are available for a possibly multinomial classification task. Specifically, ACV considers the challenging and unresolved problem of counterfeit note detection while depositing currency in an ATM that is equipped with a plurality of sensors. In an analogous manner, when predicting the structural fold of a protein multiple feature sets are available, ranging from global characteristics like the amino-acid composition and predicted secondary structure, to attributes derived from local sequence alignment such as the Smith-Waterman scores.
These problems raise the need for a classification method that is able to assess the contribution of these potentially heterogeneous object descriptors while utilizing such information to improve predictive performance.
In this talk I will present a hierarchical Bayesian multi-class multi-kernel pattern recognition machine that informatively combines the available feature groups and, as is demonstrated, is able to provide the state-of-the-art in performance accuracy on the problems considered. The full Markov chain Monte Carlo solution of the model is offered via a Metropolis-Hastings within Gibbs sampling procedure and also an efficient variational Bayes approximation is proposed.
back to seminarsJoachim Hass (University of Gottingen),Time Perception and Motion Control - Two Sides of the Same Coin?
Time is a very important dimension of our life, both for perceiving stimuli changing in time, such as speech or music, and for precisely timed coordination of movements. Currently, it is debated whether these two domains rely on common or distinct neural mechanisms. In this work, we investigate the influence of a motor task on a simultaneously performed time perception task. Our results show that this influence is not limited to a simple impairment of performance, but is specific to the state of motion the participants are in.
Participants were required to follow an elliptic trajectory that was drawn on a screen, using the end effector of a robot arm. At specific segments of this guided arm motion, namely on the apecies of the ellipse, two short tones were presented which the participants had to discriminate according to their duration. Both the discrimination performance and the perceived duration of the tones were tested. We found that the duration of the tones where systematically underestimated at the apecies where the motion was slower and the ellipse was more curved, and also overestimated in the other apecies. On the other hand, speed and curvature did not affect the discrimination performance.
These results allows for the conclusion that time perception and motor control share some neural circuitry. This correspondence can be explained in the framework of the synfire chain model, under the assumption that at least some of the chains encode both time and motion. Changing the transmission delay of these chains e.g. by altered background activity would lead to the joint change in motion speed and PSE, with little influence on the discrimination performance.
Acknowledgement: This study was supported by a grant from the Bundesministerium fuer Bildung und Forschung (BMBF) in the framework of the Bernstein Center for Computational Neuroscience Goettingen, grant number 01GQ0432.
back to seminarsJochen Steil,Online Reservoir Learning of movements for PA10 and ASIMO
We present an neural network approach for simultaneous learning of task and joint angle representations for target movements of redundant robots in a single coherent framework. For training we use an efficient online scheme for recurrent reservoir networks consisting of backpropagation-decorrelation (BPDC) output adaptation and an intrinsic plasticity (IP) reservoir optimization. We demonstrate that the network can acquire highly accurate inverse models and task predictions for the redundant 7-DOF robot arm PA-10 and the humanoid robot ASIMO, with excellent generalization to untrained target motions. The potential of the approach for imitation is shown by reproducing real data recorded from a human demonstrator. The talk will also include a short overview on recent activities in the new Bielefeld Research Institute for Cognition and Robotics (CoR-Lab) and the Excellence Cluster in Cognitive Interaction Technolgie -- (CITEC) hosting this research.
Edmond Shu Lim Ho,Synthesizing human interactions with topological constraints
In computer graphics, many researchers have been working on synthesizing realistic human animations. With the use of MOCAP systems, creating animation is no longer a time-consuming task. However, most of the existing work focus on scenes without close interactions of multiple avatars. In addition, it is difficult to capture such motions due to the limitations of the MOCAP devices.
Using individually captured motions to generate multi-character animations with close interactions is a practical approach. But it is expected that there will be a lot of collisions among the body segments of the characters. Existing methods can handle these problems by using collision detection algorithms and editing the positions of the colliding segments by inverse kinematics. However, these methods do not take into account the topological relationships between the body segments. Therefore, there is no guarantee that the topological relationships will be kept the same after the editing.
Previously we proposed a method to keep the topological constraints when editing individually captured motions. However, we do not allow the topological relationship to change. In this talk, I will present a new method to simulate the interactions between characters while satisfying the topological constraints. The new method can synthesize the transition motions between different topological states. This method is not limited to synthesizing multiple-character animations but also the interactions between character and objects with topological constraints.
back to seminarsMark Harrison,Automatically Generating Options in Reinforcement Learning
The options framework is a popular way of trying to improve the performance of reinforcement learning agents.� Options are higher-level temporally-extended macro-actions defined using a policy over lower-level actions, usually intended to achieve some specific sub-goal within the task.� Using options allows us to pre-supplying an agent with a selection of task-specific skills, hopefully improving it's performance within a problem.� However, it can be difficult and time consuming to manually construct options for a task, so a natural extension is to try and have the agent generate it's own options as it explores the environment.� I will present some of the the main current methods of generating options on-line, showing that using such approaches can improve the performance of agents in simple test problems.� I will also talk about some of the weaknesses of these approaches, and possible avenues for improvement.
back to seminarsIoannis Havoutis,Learning global control strategies for dynamically dexterous robots
One of the major outstanding issues in robotics is the design of planning and control strategies that enable robots to be both flexible and robust. In other words, we want robots that are capable of performing numerous variations on highly dexterous tasks, under a variety of different environmental conditions. This brings up the need for global control strategies - and the focus shifts towards global realizability and lazy optimization in a somewhat adversarial environment. I will present some previous work on global control that inspires my research and identify some open questions in this area. Primarily, there is a need for efficient algorithmic techniques that can acquire three things from active exploration and limited skilled demonstration - local behaviors, their regions of applicability and a specification for composition of behaviors. This way we can build on simple ideas of local -skill level- behaviors and global -task level- control, suitable for high-level planning. I will illustrate my approach with a simple weakly actuated pendulum balancing task and then outline my current and future research efforts with the KHR-2HV humanoid in simulation.
back to seminarsAdrian Haith,A Bayesian Model of Multimodal Motor Adaptation
Sensorimotor adaptation is a process which is able to utilize observations from multiple modalities to improve performance. Previous computational models of motor adaptation have tended to neglect the role of proprioception, while sensory integration and recalibration have primarily been considered in the context of passive perception and not in the context of active goal-directed movements. I will present a unified model of multimodal motor adaptation to visuomotor disturbances in which learning is driven by optimal Bayesian inference of miscalibrations in visual and proprioceptive modalities. This model is able to accurately account for the time-course of visuomotor adaptation, as well as the shifts in visual and proprioceptive perception which occur as after effects.
back to seminarsTheo Gonos,Using Homeostatic Neurons for Sensor Self-calibration
Sensor array calibration is a major problem in engineering, to which a biological approach may provide alternative solutions. I will present a homeostatic algorithm for self-calibration of individual sensory neurons. The algorithm adjusts each sensor gain and offset by trying to maintain a constant spiking activity. This mechanism was tested on the proximity sensors of a robot performing obstacle avoidance behaviour. I will show how the robot adapts to different environments using the slow time scale effects of the homeostatic calibration, e.g. adjusting sensitivity to get through gaps in cluttered environments, changing responsiveness for sensors that do not contribute to the task, and producing new responses that prevent the robot from getting stuck in one behavioural cycle.
back to seminarsBill Lewinger,Borrowing from Nature: Insect-like Robotics, Behaviour and Functionality
Insects are amazingly agile creatures capable of navigating difficult terrain with ease. By borrowing from nature, we can design biologically-inspired robots that benefit from insect evolution to accomplish similar tasks. This talk will feature past and present research in legged robotics regarding the Biologically-Inspired Legged Locomotion Ant robot (BILL-Ant), and additional control autonomy for the biologically abstracted Whegs(tm) robotic platform.
back to seminarsHubert Shum,Synthesizing Dense Interactions for Multiple Avatars
In this seminar, we propose a new method to generate a realistic scene of avatars densely interacting in a competitive or cooperative environment. Using our method, the interactions among hundreds of avatars can be created in real-time. The motions of the avatars are considered to be captured individually, which will increase the easiness of obtaining the data, while the interactions of multiple characters are synthesized using artificial intelligence and machine learning algorithms.
We first present an approach called temporal expansion, which is used to predict the future state of interaction by expanding the game tree of two avatars. Then, using this method, we efficiently sample the high dimensional state space of interaction by exploring the subspace of meaningful interactions and favouring samples that have high connectivity with the others. Using the collected samples, we create the Interaction Graph, which is a finite state machine to simulate controllable continuous interaction. We also create Interaction Patches, which can be spatio-temporally concatenated to generated animation of an interacting crowd up to hundreds of avatars.
The proposed method is superior to previous motion synthesis techniques due to its power to simulate realistic interactions. On the other hand, the ability to plan the movements of an interacting crowd creates a much more realistic crowd of avatars when comparing to tradition crowd simulation approaches.
back to seminarsSebastian Bitzer,Synthesising Novel Movements through Latent Space Modulation of Scalable Control Policies
We propose a novel methodology for learning and synthesising whole classes of high dimensional movements from a limited set of demonstrated examples that satisfy some underlying 'latent' low dimensional task constraints. We employ non-linear dimensionality reduction to extract a canonical latent space that captures some of the essential topology of the unobserved task space. In this latent space, we identify suitable parametrisation of movements with control policies such that they are easily modulated to generate novel movements from the same class and are robust to perturbations. We evaluate our method on controlled simulation experiments with simple robots (reaching and periodic movement tasks) as well as on a data set of very high-dimensional human (punching) movements. We verify that we can generate a continuum of new movements from the demonstrated class from only a few examples in both robotic and human data.
back to seminarsPaolo Favaro,Boosting Invariance and Efficiency in Supervised Learning
In this presentation I will introduce a novel boosting algorithm for supervised learning that incorporates invariance to data transformations and has high generalization capabilities. While one can incorporate invariance by adding virtual samples to the dataset (e.g., via jittering), we adopt a more efficient strategy and work along the lines of vicinal risk minimization and tangent distance methods. As in vicinal risk minimization, we incorporate invariance by introducing anisotropic smoothing along the directions of invariance. Furthermore, as in the tangent distance method, we provide a simple local approximation to such directions, so as to obtain an efficient computational scheme. Finally, we show that it is possible to automatically design optimal weak classifiers by using gradient descent. To increase efficiency at run-time, such optimal classifiers are projected onto a Haar wavelet basis. This procedure results in designing strong classifiers that are more computationally efficient than the case of exhaustive search. For illustration and validation purposes, we demonstrate the proposed algorithm on both synthetic and real data sets that are publicly available.
back to seminarsChristoph Kolodziejski,Closed loop control and behavioral learning using a differential Hebbian framework
In classical conditioning, an agent correlates an initially neutral with a behavioral relevant stimulus. This has been modeled by differential Hebbian learning. Most often that was done in an "open-loop" fashion, hence where behavior has no effect on learning. However, for an autonomous agent, its behavior will have an influence on the stimuli which it encounters and, as a consequence, also on learning. This can be emulated by closed-loop ISO-learning, an augmentation of differential Hebbian learning, where the output of the Hebbian neuron controls the agent and the loop is been closed through the environment. Here we will present an overview of the ISO-learning rule and its siblings in the context of neurophysiology as well as control theory. We will provide theoretical arguments that learning and behavior are stable, which will then also be demonstrated in several robotics applications presented in the second part of this presentation.
back to seminarsPoramate Manoonpong,Neural control for locomotion of walking machines
The basic locomotion and rhythm of stepping in walking animals like cockroaches mostly relies on a central pattern generator (CPG), while their peripheral sensors are used to control walking behaviors. By contrast, in stick insects, sensory feedback serving as reflexive mechanism plays a critical role in shaping the motor pattern for adaptivity and robustness of walking gaits. Inspired by the principles of biological locomotion control, two different types of neural mechanism for locomotion control of walking machines are presented. One is called modular reactive neural control and the other is adaptive reflex neural control. Modular reactive neural control based on a modular structure design applies a CPG for basic rhythmic leg movements and motor coordination of the six-legged walking machine AMOS-WD06 while peripheral sensors of it serve to stimulate a variety of reactive behaviors, like self-protection reflex, obstacle avoidance, phototropism, sound tropism, and wind-evoked escape behaviors. On the other hand, adaptive reflex neural control is based on reflex mechanisms. It uses proprioceptors, e.g., foot contact and angle sensors, to drive dynamic walking patterns, to regulate walking speed of the biped robot "RunBot", and also to synchronize its components. In addition, learning mechanisms have been integrated, the result of which enables RunBot to perform fast dynamic walking and autonomously learn to adapt its locomotion to different terrains.
back to seminars