Facial Expressions for Learning by Communication

Project Leaders: Marc Hanheide, Heiko Wersing
PhD Student: Christian Lang

Communication is a complex social process, prone to perception errors and demands for the constant balance between explicitness and efficiency to account for different knowledge and expectations of the interlocutors. In order to achieve communicative success appropriate feedback cues are necessary, especially in learning scenarios to structure tutoring instructions and to assure a common ground between tutor and learner. In human-human interaction these are not only verbal, but also non-verbal, e.g. gestures, body posture, and facial expressions. The latter non-verbal cue - facial expressions and their dynamics - is in the focus of this project. The hypothesis is that facial expressions provide a means to facilitate dialog management on different levels as known from linguistics:

• turn-taking (e.g. "freeze" after statement waiting for a response)
• implicit grounding-related feedback (e.g. looking puzzled, raise eyebrows)
• implicit evaluation of the situation (e.g. looking unpleased, satisfied or surprised)

The embedding of facial expressions and dynamics will ease social learning in terms of an emphasis of the structure of tutoring discourses as also to serve reinforcement signals. Application scenarios range from human-robot interaction to analysis of human-human tutoring studies. The goal is to reveal the potential of analyzing human social interaction using facial expressions and their dynamics to support learning by communicating.