Cognitive Planning and Motor Adaptation in Manual Action

Project Leaders: Thomas Schack, Yaochu Jin
Postdoctoral Associate: Christian Seegelke
PhD Student: Heiko Lex

The development of appropriate representations for manual action episodes and their coordination to more complex action sequences still poses a major challenge for robotics. Hence, a natural approach is to gain insight to these questions from a cognitive and experimental perspective revealing the development of sensorimotor representations in humans accomplishing those complex tasks. Especially, it is interesting to learn about the relationship between the structure of representations and the performance in the execution of sensorimotor tasks, like sequential object manipulations.

As the production of manual actions in humans is affected by a number of factors, such as biomechanical constraints (e.g. Weigelt et al. 2006) and action representations (Schack, 2004), this project focuses on the question of how structures of sensorimotor representations are established and gradually changed, while considering the physical properties of objects, task constraints (affordances) and perceptual discordances in sensorimotor adaptation tasks. It is of interest to learn about (A) the relationship between the structure of representations and the performance in sensorimotor adaptation tasks and (B) performance in the planning of complex object manipulations, while focusing on sequential behavior. Both cases, sensorimotor adaptation to changing environmental conditions and the sequential organization of grasping behavior, are situations in which action can result in error. Hence, one of the most interesting questions is what kind of representation helps people to control actions and to learn from errors.

To this end, two different types of experimental tasks will be created. The first type will be applied to investigate (sensori-) motor adaptation to changing conditions (i.e. transformation of visual and kinesthetical feedback information), the second type to look at the sequential organization and adaptation of grasping movements (i.e. as assessed by adaptation / changes over time). Additionally, we will apply experimental methods to measure representation structures concerning objects and grasping movements in long-term memory and in working memory, as well as measures of the representation of movement direction. The broader aim of the project is to combine these tasks and experimental methods to find out more about the cognitive modules enabling (sensori-) motor adaptation to environmental changes and the cognitive planning of manual action. In a last step we are searching for a translation of cognitive representation and behavioral data in humans in a technical system (Artificial Neural Networks). We are going to produce Artificial Neural Networks (Self-organizing Maps and Recurrent Networks) which includes (saves) the functional information of human kinematics and representation.

On the one hand, this project can give valuable input to the planning project on the robotics side, regarding basic structural principles to organise complex action patterns. On the other hand, it opens the opportunity to verify assumptions made for the planning model on an experimental basis.