Short Description

Spoken face-to-face communication is a central component of interaction with social robots in the future. Furhat, developed at the Stockholm-based startup Furhat Robotics, is one of the most advanced social robots. In order to communicate with humans, it makes use of information from spoken information and of visual information in the form of facial expressions, lip movement and gaze. The robot already has a very natural behaviour, but for now the robot only responds when the user has finished speaking, which is not the case with most human-human conversations work. There is room to improve this by adding Backchannels in the robot’s repertoire of possible responses, e.g. to say something like “mhm, aha, a jo” at the right moment in order to be perceived by the human as a more authentic listener. This project consists of two parts concerning these Backchannels:

1) The GRASS corpus consists of spontaneous speech in which Hearer Response Tokens (HRT, Backchannels) and the communicative functions that trigger the HRTs are annotated. These HRTs and communicative functions should be analysed in different ways (prosody, timing, features in general, …). The collected information should then be used for the second part of the project.

2) To make the robot more natural, a backchannel model should be implemented in the furhat robot. With the information gathered in the corpus study and information from the literature, the robot should learn how and when to make backchannels. This model should be evaluated in terms of the naturalness of the conversation with the robot.

Tasks

• Reading into GRASS corpus and communicative functions
• Extracting backchannel-tokens (HRTs, etc.)
• Analysis of tokens
• Developing Backchannel model for Furhat
• Evaluation of models
• Reporting results (thesis writing)

Profile

• basic knowlegde of sound engineering and/or speech communication
• good knowledge of programming in Python
• basic knowledge of Furhat and Kotlin is an advantage