Article | Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication, August 6-8, 2014, Tartu, Estonia | On the Attribution of Affective-Epistemic States to Communicative Behavior in Different Modes of Recording
Göm menyn

Title:
On the Attribution of Affective-Epistemic States to Communicative Behavior in Different Modes of Recording
Author:
Stefano Lanzini: SCCIIL (SSKKII), Gothenburg University, Sweden Jens Allwood: SCCIIL (SSKKII), Gothenburg University, Sweden
Download:
Full text (pdf)
Year:
2015
Conference:
Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication, August 6-8, 2014, Tartu, Estonia
Issue:
110
Article no.:
007
Pages:
47-52
No. of pages:
6
Publication type:
Abstract and Fulltext
Published:
2015-05-26
ISBN:
978-91-7519-074-7
Series:
Linköping Electronic Conference Proceedings
ISSN (print):
1650-3686
ISSN (online):
1650-3740
Publisher:
Linköping University Electronic Press, Linköpings universitet


Export in BibTex, RIS or text

Face-to-face communication is multimodal with varying contributions from all sensory modalities, see e.g. Kopp (2013), Kendon (1980) and Allwood (1979). This paper reports a study of respondents inter- preting vocal and gestural verbal and non-verbal, behavior. 10 clips from 5 different short video + audio recordings of two persons meeting for the first time were used as stimulus in a perception/classification study. The respondents were divided in 3 different groups. The first group watched only the video part of the clips without any sound. The second group listened to the audio track without video. The third group was exposed to both the audio and video tracks of the clip. In order to collect the data, we used a crowdsourcing questionnaire. The study reports on how respondents classified clips containing 4 differ- ent types of behavior (looking up, looking down, nodding and laughing) that were found to be frequent in a previous study (Lanzini 2013) according to which Affective Epistemic State (AES) the behaviors were perceived as expressing. We grouped the linguistic terms for the affective epistemic states that the respondents used in- to 27 different semantic fields. In this paper we will focus on the 7 most common fields, i.e. the fields of Thinking, Nervousness, Happiness, Assertiveness, Embarrassment, Indifference and Interest. The aim of the study is to increase understanding of how exposure to video and/or audio modalities affect the interpretation of vocal and gestural verbal and non-verbal behavior, when they are displayed uni- modally and multi-modally.

Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication, August 6-8, 2014, Tartu, Estonia

Author:
Stefano Lanzini, Jens Allwood
Title:
On the Attribution of Affective-Epistemic States to Communicative Behavior in Different Modes of Recording
References:

Allwood, J. 1979."Ickeverbal kommunikation - en översikt" in Stedje and af Trampe (Ed.) Tvåspråkighet. Stockholm, Akademilitteratur. Also in Invandrare och Minoriteter nr 3, 1979, pp. 16-24.


Allwood, J. and Cerrato, L. 2003 A Study of Gestural Feedback Expressions. First Nordic Symposium on Multimodal Communication. Paggio P. Jokinen, K. Jönsson, A. (eds). Copenhagen, 23-24, September 2003, pp. 7-22.


Boholm M. and Lindblad G. 2011. Head movements and prosody in multimodal feedback. NEALT Proceedings Series: 3rd Nordic Symposium on Multimodal Communication, 15, p. 25-32.


Chindamo, Massimo, Allwood, Jens & Ahlsén, Elisabeth. 2012. Some suggestions for the study of stance in communication. Proceedings of IEEE SocialCom Amsterdam 2012, 3-5.


Kopp Stefan. 2013. Giving interaction a hand: deep models of co-speech gesture in multimodal systems. Proceedings of the 15th ACM on International conference on multimodal interaction. ACM, 245-246.


Kendon,Adam. 1980. Gesticulation and speech: two aspects of the process of utterance. In M.R.Key (ed), The Relationship of Verbal and Nonverbal Communication, pp. 207-227. The Hague: Mouton and Co.


Lanzini, Stefano. 2013. How do different modes contribute to the interpretation of affective epistemic states. Published master’s thesis for master’s degree. University Gothenburg, Division of Communication and Cognition, Department of Applied IT.


Schroder Marc, Bevacqua Elisabetta, Cowie Roddy, Eyben Florian, Gunes Hatice, Heylen Dirk, ter Maat Mark, McKeown Gary, Pammi Sathish, Pantic Maja, Pelachaud Catherine, Schuller Bjorn, de Sevin Etienne, Valstar Michel, and Wollmer Martin. 2011. Building Autonomous Sensitive Artificial Listeners. IEEE Trans. Affective Computing. 9. (1). p. 1.

Proceedings of the 2nd European and the 5th Nordic Symposium on Multimodal Communication, August 6-8, 2014, Tartu, Estonia

Author:
Stefano Lanzini, Jens Allwood
Title:
On the Attribution of Affective-Epistemic States to Communicative Behavior in Different Modes of Recording
Note: the following are taken directly from CrossRef
Citations:
No citations available at the moment


Responsible for this page: Peter Berkesand
Last updated: 2017-02-21