A Taxonomy of Social Cues for Conversational Agents

When using the taxonomy, please cite as Feine, J., Gnewuch U., Morana S. and Maedche A. (2019): “A Taxonomy of Social Cues for Conversational Agents” International Journal of Human-Computer Studies. To read the paper, please click here.

Social cue: Posture shift
Communication system: Visual
Cue category: Kinesics
Cue Description
The CA moves its upper or lower body.
Cue example
CA turns its body.
Cue impact
Posture shifts occur when the agent is moving his body towards a specific direction. Channel of nonverbal communication in face-to-face dialogues in order to initiate and terminate a conversation (Cassel, Bickmore 2000). Besides others, it further elicits more social behavior of the user (Appel et al. 2012), and is useful as part of relational behavior strategies to ensure a long-term working alliance (Bickmore et al. 2005). Turning towards the user is further an acknowledgement of the user’s presence (Cassel et al. 1999). Furthermore, certain body postures can create the impression of negative or positive evaluation by the agent which elicit fear and anxiety (Pertaub et al. 2001).Compared to active feedback behavior, simple posture shifts do not create enough behavioral realism (Von der Pütten et al. 2009) to build rapport (Kraemer et al. 2016) or to increase the perceived social presence and amount of spoken words (Puetten et al. 2010). Breathing as part of other secondary behaviors creates the impression that the agent is being alive (Kraemer et al. 2016; Becker et al. 2005), believable (Becker et al. 2004) and elicits more social behavior of the user (Appel et al. 2012). However, it does not enhance the perceived social presence like reactive nonverbal feedback (Puetten et al. 2010; Von der Pütten et al. 2009).
Reference List
1. Appel, J., Pütten, A. von der, Krämer, N. C., & Gratch, J. (2012). Does Humanity Matter?: Analyzing the Importance of Social Cues and Perceived Agency of a Computer System for the Emergence of Social Reactions during Human-Computer Interaction. Advances in Human-Computer Interaction (2012:2), pp. 1-10.
2. Bickmore, T. W., & Picard, R. W. (2005). Establishing and Maintaining Long-term Human-computer Relationships. ACM TRANSACTIONS ON COMPUTER-HUMAN INTERACTION (12:2), pp. 293-327, from http://doi.acm.org/10.1145/1067860.1067867.
3. P Pertaub, D., Slater, M., & Barker, C. (2001). An experiment on fear of public speaking in virtual reality. Studies in health technology and informatics (81:
4. Becker, C., Kopp, S., & Wachsmuth, L. (2004). Simulating the emotion dynamics of a multimodal conversational agent. In E. Andre, L. Dybkjaer, W. Minker, & P. Heisterkamp (Eds.): Lecture Notes in Computer Science, AFFECTIVE DIALOGUE SYSTEMS, PROCEEDINGS (pp. 154-165).
5. Becker, C., Prendinger, H., Ishizuka, M., & Wachsmuth, I. (Eds.). 2005. Evaluating affective feedback of the 3D agent max in a competitive cards game: Springer.
6. Cassell, J., & Bickmore, T. (2000). EXTERNAL MANIFESTATIONS OF TRUST WORTHINESS IN THE INTERFACE. Communications of the ACM (43:12), pp. 50-56.
7. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., & Yan, H. (1999). Embodiment in Conversational Interfaces: Rea. In : CHI ’99, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 520-527). New York, NY, USA: ACM.
8. Kraemer, N. C., Karacora, B., Lucas, G., Dehghani, M., Ruether, G., & Gratch, J. (2016). Closing the gender gap in STEM with friendly male instructors? On the effects of rapport behavior and gender of a virtual agent in an instructional interaction. COMPUTERS & EDUCATION (99, pp. 1-13.
9. Puetten, A. M. von der, Kraemer, N. C., Gratch, J., & Kang, S.-H. (2010). “It doesn’t matter what you are!” Explaining social effects of agents and avatars. Computers in Human Behavior (26:6), pp. 1641-1650.
10. von der Pütten, Astrid Marieke, Krämer, N., & Gratch, J. (2009). Who s there? Can a Virtual Agent Really Elicit Social Presence? The 12th Annual International Workshop on Presence.
11. Cassell, J. (2000). Embodied conversational interface agents? Communications of the ACM 43 (4), 70?78.
12. Cassell, J. and K. R. Thorisson (1999). ?The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents? Applied Artificial intelligence 13 (4-5), 519?538.
13. Bickmore, T. and J. Cassell (2005). Social Dialogue with Embodied Conversational Agents. In J. C. J. Kuppevelt, N. O. Bernsen and L. Dybkjær (eds.) Advances in Natural Multimodal Dialogue Systems, pp. 23?54. Dordrecht: Springer.
14. Thiebaux, M., S. Marsella, A. N. Marshall and M. Kallmann (2008). Smartbody: Behavior realization for embodied conversational agents. In International Foundation for Autonomous Agents and Multiagent SystemsInternational Foundation for Autonomous Agents and Multiagent Systems.
15. Kopp, S. and I. Wachsmuth (2004). Synthesizing multimodal utterances for conversational agents Computer animation and virtual worlds 15 (1), 39?52.
16. Cassell, J. (2001). Embodied Conversational Agents. Representation and Intelligence in User Interfaces AI MAGAZINE 22 (4), 67?83.