Evidence for Communicative Compensation in Debt Advice with Reduced Multimodality

Nicole Andelic, Aidan Feeney, Gary McKeown

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

1 Citation (Scopus)


Research has found that professional advice with empathy displays and signs of listening lead to more successful outcomes. These skills are typically displayed through visual nonverbal signals, whereas reduced multimodal contexts have to use other strategies to compensate for the lack of visual nonverbal information. Debt advice is often a highly emotional scenario but to date there has been no research comparing fully multimodal with reduced multimodal debt advice. The aim of the current study was to compare explicit emotional content (as expressed verbally) and implicit emotional content (as expressed through paralinguistic cues) in face to face (FTF) and telephone debt advice recordings. Twenty-two debt advice recordings were coded as emotional or functional and processed through emotion recognition software. The analysis found that FTF recordings included more explicit emotion than telephone recordings did. However, linear mixed effects modelling found substantially higher levels of arousal and slightly lower levels of valence in telephone advice. Interaction analyses found that emotional speech in FTF advice was characterised by lower levels of arousal than during functional speech, whereas emotional speech in telephone advice had higher levels of arousal than in functional speech. We can conclude that there are differences in emotional content when comparing full and reduced multimodal debt advice. Furthermore, as telephone advice cannot avail of visual nonverbal signals, it seems to compensate by using nonverbal cues present in the voice.
Original languageEnglish
Title of host publicationICMI '19 2019 International Conference on Multimodal Interaction
EditorsWen Gao, Helen Mei Ling Meng, Matthew Turk, Susan R Fussell, Bjorn Schuller, Yale Song, Kai Yu
Place of PublicationNew York
PublisherAssociation for Computing Machinery (ACM)
Number of pages10
ISBN (Electronic)9781450368605
ISBN (Print)978-1-4503-6860-5
Publication statusPublished - 14 Oct 2019
Event21st ACM International Conference on Multimodal Interaction
- Suzhou, Jiangsu, China
Duration: 14 Oct 201918 Oct 2019


Conference21st ACM International Conference on Multimodal Interaction
Abbreviated titleICMI
CitySuzhou, Jiangsu


  • Advice-giving
  • Affective computing
  • Arousal
  • Valence


Dive into the research topics of 'Evidence for Communicative Compensation in Debt Advice with Reduced Multimodality'. Together they form a unique fingerprint.

Cite this