A Taxonomy of Social Cues for Conversational Agents

When using the taxonomy, please cite as Feine, J., Gnewuch U., Morana S. and Maedche A. (2019): “A Taxonomy of Social Cues for Conversational Agents” International Journal of Human-Computer Studies. To read the paper, please click here.

Social cue: Vocal segregate
Communication system: Auditory
Cue category: Vocalizations
Cue Description
The CA makes sounds that get in the way of fluent speech.
Cue example
CA: "uhs", "ums", stuttering.
Cue impact
Hesitation (mhm, eh, uhm etc.) sounds enable the agent to answer the user before the complete message is transcribed and are perceived as more polite, more efficient, better at indicating when to speak (Skantze, Hjalmarsson 2013), and can be used to indicate understanding (Cassel et al. 1999). Another study revealed mixed evaluation results ranging from inappropriate behavior to a more humanized experience (Pfeifer, Bickmore 2009). Non-fluencies portray a lack of expertise (Cowell, Stanney 2005).
Reference List
1. Cowell, A. J., & Stanney, K. M. (2005). Manipulation of non-verbal interaction style and demographic embodiment to increase anthropomorphic computer character credibility. INTERNATIONAL JOURNAL OF HUMAN-COMPUTER STUDIES (62:2), pp. 281-306.
2. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjálmsson, H., & Yan, H. (1999). Embodiment in Conversational Interfaces: Rea. In : CHI ’99, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 520-527). New York, NY, USA: ACM.
3. Pfeifer, L. M., & Bickmore, T. (2009). Should Agents Speak Like, Um, Humans? The Use of Conversational Fillers by Virtual Agents. In : IVA ’09, Proceedings of the 9th International Conference on Intelligent Virtual Agents (pp. 460-466). Berlin, Heidelberg: Springer-Verlag.
4. Skantze, G., & Hjalmarsson, A. (2013). Towards incremental speech generation in conversational systems. Computer Speech & Language (27:1), pp. 243-262.
5. Cassell, J. (2000). Embodied conversational interface agents? Communications of the ACM 43 (4), 70?78.
6. Bickmore, T. and J. Cassell (2005). Social Dialogue with Embodied Conversational Agents. In J. C. J. Kuppevelt, N. O. Bernsen and L. Dybkjær (eds.) Advances in Natural Multimodal Dialogue Systems, pp. 23?54. Dordrecht: Springer.
7. Cassell, J. (2001). Embodied Conversational Agents. Representation and Intelligence in User Interfaces AI MAGAZINE 22 (4), 67?83.
8. Bevacqua, E., S. Pammi, S. J. Hyniewska, M. Schröder and C. Pelachaud (2010). Multimodal Backchannels for Embodied Conversational Agents. In: Intelligent Virtual Agents. Ed. by J. Allbeck, N. Badler, T. Bickmore, C. Pelachaud, A. Safonova. Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 194?200.