ACII 2021 - 1st Workshop on

Functions of Emotions for Socially Interactive Agents

What is it about ...


The workshop takes place today in the ACII’s Gather.Town platform


We might smile because we are indeed happy because we want to build a pleasant connection to the other, or maybe because we want to hide shame and insecurity. We might enjoy the company of Paula very much, or we might be a bit unrelaxed because she reminds us of our mother-in-law. When we grew up in the US, we might show our disgust about something obvious, but we might most likely show a polite smile if we grew up in Japan. In a discussion, we might show anger because we are indeed in a conflict with our counterpart, because of the topic we are talking about or because we are angry with ourselves. We might look sad and downhearted because we are indeed sad, or maybe, we unconsciously suppressed anger which turned into depression.

This workshop explores the challenges and the possibilities that come with the view that emotions are not universally unique patterns (internally and externally) and always connected to individual experiences. The focus is on exploring the concept of functions of emotions because we believe that this approach holds great potential for empathic systems.

Emotions have intrapersonal functions (e.g., motivational purposes, guidance of perception, and decision making), interpersonal functions (e.g., signaling of the nature of a relationship or a topic, providing incentives for specific behaviors and underline meanings, illustrating the communicated topic), and socio-cultural functions (e.g., coordinate social situations through the connection of norms to values, beliefs and behavior).

Empathic and socially interactive agents would benefit from such a view with regard to understanding the dialog partner on an emotional level and showing appropriate behavior. Such agents can play into each of these functions and purposes by, e.g., complying with suitable situational dependent behavioral norms, mirroring behavior. Relying only on the interpretation of social signals might not be enough for every social context. Contextual information, individual differences of the users, and group characteristics are indeed crucial for this process. A computational representation of functions of emotions might be the key to next-gen empathic, socially interactive agents. Foremost, a representation of emotions that integrates individual subjective experience is required. Based on such a representation, individual and social functions of emotions can be modeled. Within that context, it is of most interest how these functions can be related to observable social signals (e.g., voice, gaze, gesture, body movements) and displayed emotions between, at least, two individuals. Moreover, empathic systems in various fields of application (e.g., therapeutic assistance, autonomous driving cars, learning social skills, learning, and working in groups) could be improved by incorporating computational models for functions of emotions.

This workshop intends to bring researchers from diverse research areas together to share recent advances and discuss research directions and opportunities for the next generation of computational models and human-computer-interaction that take functions of emotions into account. We invite submissions of research papers and position papers that address the following areas (but not limited to):

  • Theoretical work on functions of emotions and their relation to 1) social signals and their exchange between humans and 2) higher-level social constructs such as trust, relationship, and cultural and social norms. Moreover, innovative or cross-disciplinary methodological approaches to research functions of emotions.
  • Computational models of functions of emotions and their relation to observable social signals as well as their use in empathic or socially interactive agents (virtual and physical).
  • Computational models of emotions that come with a representation of individual subjective experience and functions of emotions.
  • Models defining the relation of emotional functions to learning processes, feedback strategies, and their impact on the social relationship in learning contexts.
  • Empirical studies to develop or evaluate computational models considering functions of emotions.
  • Empirical studies on functions of emotions and their relation to social signals in human-human-interaction including pilot studies applying new methodological approaches.
  • Applications that employ a computational representation of functions of emotions and demonstrate the benefit of such by showing empathic behavior.


We invite submissions of research papers from authors with diverse backgrounds (e.g., computer science, psychology, sociology, learning science, behavioral science). Submissions can be on completed work, work-in-progress, position papers, posters, or demos. Authors are asked to submit papers up to 6 pages (excluding references) following the submission guidelines from the ACII 2021 conference. Please use the conference’s EasyChair system for submissions. Remeber to select the Func-E track while submitting. All submissions will be reviewed double-blind.

At least one author of each accepted paper will be required to attend the workshop to present their work.

Instructions for prerecorded 5min talks

Please include the following information:
1. Research question
2. Starting point/challenge
3. Connection/ Statement to functions of emotion
4. Method
5. Results

Do your best to create a vivid and engaging presentation, and include subtitles. Be prepared to answer live question afterwards (2-4 mins). Pump yourself up before to be energetic!

Please upload your slides and presentation here - naming schema is {your paper number}_slides.{ppt or pdf} and {your paper number}_presentation.mp4

Important Dates

  March 31, 2021: Workshop announcement

  June 14, 2021: Submission Deadline

  July 20, 2021 (changed from June 30): Notification

  August 13, 2021 (changed from July 30): Camera Ready Deadline

  September 24, 2021: Video Contribution Deadline

  September 28, 2021: Workshop


The workshop takes place in the ACII’s Gather.Town platform at September 28, 2021 from 12:00 - 17:00 CET. Individual talks are realized with 5min prerecorded videos followed by a short time for life questions and discussion between author(s) and participants.

12:00 Intro
12:15 Invited talk 1: What is that thing ‘Emotion’, Arvid Kappas
12:55 Small break
13:00 Concept-/theory-oriented talks with tiny breaks and discussions
14:00 General group discussion - Statements and what next
14:15 Big break - Bring your lunch to the garden
15:00 Invited talk 2: Functions of Emotions - The Basis of Human’s Well-being, Jana Volkert
15:40 Small break
15:45 Technical-/application-oriented talks with tiny breaks and discussions
16:35 General group discussion - Statements and what next
16:50 Wrap-up

Invited Talks

What is that Thing 'Emotion'?

Abstract: While it is well known that researchers have a hard time defining emotion, it is less well known that the cohesion between individual components of emotions, such as subjective experience, physiological activation, expressive behavior is often rather low. What happens to these components in a real situation is often shaped by facets of context and function in a given situation. Classical research on affect, 50 years ago, assumed that laboratory studies of emotions often would find only low correlations because the intensity of the emotions studied was too low. While this may or may not be true (hint: I do not think it is), it is bad news for the engineer who wants to build an algorithm to identify a specific affective state in the real world when intensities there are also often low. I will discuss the challenge of low correlations between emotional components and pose the question of whether sometimes concepts, such as actions and functions might be more fruitful for the engineer than implying that identifying specific emotions will be the best predictor of behavior, attitudes, or motivations.

Arvid Kappas is a professor of psychology and Dean at Jacobs University Bremen. He has been conducting research on emotions for over three decades. Having obtained his PhD at Dartmouth College, NH, USA, he has lived and worked in Switzerland, Canada, the UK, and in Germany. He was also visiting professor in Austria and in Italy. Between 2013 and 2018 he was the president of the International Society for Research on Emotion. Kappas is an elected fellow of the Association for Psychological Science and the Society for Personality and Social Psychology and presently a member of the executive board of the Association for the Advancement of Affective Computing. His current research interests relate to the affective in affective computing and the social in social robotics.

Functions of Emotions - the Basis of Human’s Well-being

Abstract: The ability to recognize, experience and regulate emotions are at the core of what makes us human beings. The mirroring of affects is a key component of our earliest relationships with our primary caregivers, essential for the development of secure attachment, developing one’s own identity and the ability to form and enjoy relationships with others. Furthermore, our inner capacity to recognize and regulate emotions is also related to our capacity to learn in social environments and maintain good mental health. This keynote will present current theoretical knowledge and empirical evidence about the functions of emotions for our mental health and their role in delivering effective treatments for mental health problems from a clinical psychological perspective. The great potential and challenges to use affective computational systems for promoting mental health and delivering interventions will be discussed.

Jana Volkert is is a visiting professor in clinical psychology at the University of Kassel, and a research fellow at the University of Heidelberg, Germany. Her research focuses onthe development and treatment of mental health problems in particular with regard togeneral processes of emotion regulation and mentalizing capacities in children, adolescents and adults. She obtained her PhD at the University of Hamburg (Germany), conducted her postdoctoral research fellowship at McLean Hospital and Harvard Medical School, MA, USA, and is also a licensed psychotherapist, specialized in psychodynamic and mentalization-based treatments.

Paper and Talk Overview

Concept focussed 13:00-14:00 CET
137 Aarti Malhotra and Jesse Hoey Emotions in Socio-cultural Interactive AI Agents
144 David Antonio Gómez Jáuregui, Felix Dollack and Monica Perusquía-Hernández Robot mirroring: Improving well-being by fostering empathy with an artificial agent representing the self
158 Mirella Hladký, Tanja Schneeberger and Patrick Gebhard Understanding Shame Signals: Functions of Smile and Laughter in the Context of Shame
160 Janet Wessler Economic and Social Consequences of Anger and Gender in Computer-Mediated Negotiations: Is there a Backlash Against Angry Females?
162 Thomas Kiderle, Hannes Ritschel, Kathrin Janowski, Silvan Mertes, Florian Lingenfelser and Elisabeth André Socially-Aware Personality Adaptation
173 Lara Chehayeb, Dimitra Tsovaltzi, Rhythm Arora and Patrick Gebhard Individual Differences and the Function of Emotions in Socio-Emotional and Cognitive Conflict: If an agent shames you, will you still be bored?
Technical / Application focussed 15:45 - 16:35 CET
151 Silvan Mertes, Thomas Kiderle, Ruben Schlagowski, Florian Lingenfelser and Elisabeth André On the Potential of Modular Voice Conversion for Virtual Agents
157 Kazuhiro Shidara, Hiroki Tanaka, Hiroyoshi Adachi, Daisuke Kanayama, Yukako Sakagami, Takashi Kudo and Satoshi Nakamura Relationship between Mood Improvement and Questioning to Evaluate Automatic Thoughts in Cognitive Restructuring with a Virtual Agent
159 Laduona Dai and Joost Broekens Simulating Fear as Anticipation of Temporal Differences: an experimental investigation
161 Hannes Ritschel, Thomas Kiderle and Elisabeth André Implementing Parallel and Independent Movements for a Social Robot's Affective Expressions
179 Jacky Casas, Samuel Torche, Karl Daher, Elena Mugellini and Omar Abou Khaled Emotional Paraphrasing using Pre-trained Language Models

Organizers and Committee

Programm Committee

Michael Baker



CNRS, Télécom ParisTech, France

Tobias Baur



Augsburg University, Germany

Joost Broekens



University of Leiden, Netherlands

Joana Campos



INESC-ID, Portugal

Celine Clavel



LIMSI Paris, France

Oliver Evers



University Hospital, Heidelberg, Germany

Arthur Graesser



University of Memphis, USA

Rachael Jack



University Glasgow, Scotland, UK

Malte Jung



Cornell University, USA

Nicole Krämer



University Duisburg-Essen, Germany

Birgit Lugrin



Würzburg University, Germany

Jean-Claude Martin



LIMSI Paris, France

Prasanth Murali



Northeastern University, USA

Philipp Müller



DFKI, Germany

Piia Näykki



University of Jyväskylä, Finland

Fabrizio Nunnari



DFKI, Germany

Magalie Ochs



Laboratoire d’Informatique et des Systèmes, France

Catherine Pelachaud



CNRS - ISIR, Sorbonne University, France

Paolo Petta



ÖFAI, Austria

Claire Polo



Lyon 2 University - ECP Laboratory, France

Brian Ravenet



IMSI Paris, France

Nicolas Sabouret



LIMSI Paris, France

Clara Falala-Séchet



University Paris Descartes, France

Hiroki Tanaka



RIKEN Center for Advanced Intelligence, Japan

Ilaria Torre



KTH Royal Institute of Technology, Sweden

Christiana Tsiourti



Messerli Research Institute, Austria

Katharina Weitz



Augsburg University, Germany

Janet Wessler



Saarland University, Germany

Nutchanon Yongsatianchot



Northeastern University, USA