How I became involved in social robotics

Some say that conversation is an art. When you try to build an artificial agent capable of even a limited form of social conversation, you begin to understand what people are getting at. In 2008 I was employed as a RA/developer at Oxford University to work on the EU funded Companions Project, which sought to develop an animated avatar called Samuela that you could have a 'social' conversation with about your day at work.

Samuela was designed to be emotionally intelligent, recognising the user's emotional state through voice patterns and sentiment analysis, and using her voice, facial expressions and gestures to show empathy towards the user. She was also capable of generating long utterances in which she gave advice to the user about how they were responding emotionally to the events of their day. Here is a video which introduces the prototype system and shows a couple of the couple of sample conversations with a user:

If you want to know more of the technical details of the system have a look at the selected references listed below. I will summarise my contributions to the Companions project in following Blog posts.

Working on this project was one of the most exciting and challenging periods of my research career. It introduced me to the deeply interesting and challenging area of creating artefacts capable of social interaction with people. Such systems require us to go far beyond the traditional mainstream challenges of AI (e.g. NLP, reasoning, learning, dialog management etc), into a world dominated by social norms and protocols, emotion, ethical patterns of behaviour and much more. I also realised the importance of 'presence' in social interaction, and in particular, bodily presence. An avatar on a screen (just like a human on a screen) involves a certain remoteness and lack of presence. For this reason, I turned to the use of robots to study and develop technologies capable of social interaction with people.

In 2011 I was appointed as Head of Computing and Communication Technologies at Oxford Brookes University. Soon afterwards I opened a new Cognitive Robotics lab there and began work on producing robots capable of social interaction, including our own skeletal head-and-neck robot called Eddie (more about this in later Blog posts) which we have built to mimic human head movements during conversation. We have also recently completed a study of the effect that upper body pose mirroring has on human-robot interaction. In this series of blog posts I will summarise this and subsequent work and give some insights into the stories behind the publications.

References

Cavazza, M.,  Santos de la Cámara, R.,  Turunen, M.,  Gil, J.,  Hakulinen, J.,  Crook, N.T. and Field, D. (2010)  ‘How was your day?’ An Affective Companion ECA Prototype. In: Proceedings of the 11th Annual SIGdial Meeting on Discourse and Dialogue. Tokyo, Japan. 24th – 25th September 2010.

Crook, N.T., Smith, C., Field, D., Cavazza, M., Pulman, S., Moore, R., and Boye, J. (2012) Generating context-sensitive ECA responses to user barge-in interruptions. Journal of Multimodal User Interfaces, 6 (1), 13-25.

Hakulinen,J, Turunen, M, Santos de la Camara, S. and Crook, N.T. (2010) Parallel Processing of Interruptions and Feedback in Companions Affective Dialogue System. In: Proceedings of Interspeech2010, Makuhari, Japan. 26-30 September 2010.

Smith, C., Crook, N.T., Boye, J., Charlton, D., Pizzi, D., Cavazza, M. and Pulman, S. (2011) Interaction Strategies for an Affective Conversational Agent. Presence, October 2011, Vol. 20, No 5, 395-411.