Article | Proceedings from the 3rd European Symposium on Multimodal Communication, Dublin, September 17-18, 2015 | Coordination of Head Movements and Speech in First Encounter Dialogues
Göm menyn

Title:
Coordination of Head Movements and Speech in First Encounter Dialogues
Author:
Patrizia Paggio: University of Copenhagen, Denmark / University of Malta, Malta
Download:
Full text (pdf)
Year:
2015
Conference:
Proceedings from the 3rd European Symposium on Multimodal Communication, Dublin, September 17-18, 2015
Issue:
105
Article no.:
012
Pages:
69-74
No. of pages:
6
Publication type:
Abstract and Fulltext
Published:
2016-09-16
ISBN:
978-91-7685-679-6
Series:
Linköping Electronic Conference Proceedings
ISSN (print):
1650-3686
ISSN (online):
1650-3740
Publisher:
Linköping University Electronic Press, Linköpings universitet


Export in BibTex, RIS or text

This paper presents an analysis of the temporal alignment be- tween head movements and associated speech segments in the NOMCO corpus of first encounter dialogues [1]. Our results show that head movements tend to start slightly before the onset of the corresponding speech sequence and to end slightly after, but also that there are delays in both directions in the range of -/+ 1s. Various factors that may influence delay duration are investigated. Correlations are found between delay length and the duration of the speech sequences associated with the head movements. Effects due to the different head movement types are also discussed.

Keywords: head movements, movement-speech alignment, delays, dialogues

Proceedings from the 3rd European Symposium on Multimodal Communication, Dublin, September 17-18, 2015

Author:
Patrizia Paggio
Title:
Coordination of Head Movements and Speech in First Encounter Dialogues
References:

[1] P. Paggio, J. Allwood, E. Ahls´en, K. Jokinen, and C. Navarretta, “The NOMCO multimodal nordic resource - goals and characteristics,” in Proceedings of the Seventh conference on International Language Resources and Evaluation (LREC’10). Valletta, Malta: European Language
Resources Association (ELRA), 2010.


[2] D. McNeill, Hand and Mind: What Gestures Reveal About Thought. University of Chicago Press, Chicago, 1992.


[3] D. McNeill, Gesture and thought. University of Chicago Press, 2005.


[4] A. Kendon, Gesture: Visible Action as Utterance. Cambridge University Press, 2004.


[5] S. Kita and A. O¨ zyu¨rek, “What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking,” Journal of Memory and Language, vol. 48, no. 1, pp. 16–32, 2003.


[6] J.-P. De Ruiter, “The production of gesture and speech,” in Language and Gesture. Cambridge University Press, 2000.


[7] D. Bolinger, Intonation and its parts: Melody in spoken English. CA: Stanford: Stanford, 1986.


[8] A. Kendon, “Gesture and speech: two aspects of the process of utterance,” in Nonverbal Communication and Language, M. R. Key, Ed. Mouton, 1980, pp. 207–227.


[9] D. P. Loehr, “Gesture and intonation,” Ph.D. dissertation, Georgetown University, 2004.


[10] D. P. Loehr, “Aspects of rhythm in gesture and speech,” Gesture, vol. 7, no. 2, 2007.


[11] T. Leonard and F. Cummins, “The temporal relation between beat gestures and speech,” Language and Cognitive Processes, vol. 26, no. 10, pp. 1457–1471, 2010.


[12] G. Giorgolo and F. A. Verstraten, “Perception of ‘speechand-gesture’ integration,” in Proceedings of the International Conference on Auditory-Visual Speech Processing 2008, 2008, pp. 31–36.


[13] U. Hadar, T. Steiner, E. C. Grant, and F. C. Rose, “Head movement correlates of juncture and stress at sentence level,” Language and Speech, vol. 26, no. 2, pp. 117–129, 1983.


[14] U. Hadar, T. Steiner, and F. C. Rose, “The timing of shifts of head postures during conversation,” Human Movement Science, vol. 3, no. 3, pp. 237–245, 1984.


[15] M. Kipp, Gesture Generation by Imitation – From Human Behavior to Computer Character Animation. Boca Raton, Florida: Dissertation.com, 2004.


[16] C. Navarretta and P. Paggio, “Verbal and Non-Verbal Feedback in Different Types of Interactions,” in Proceedings of LREC 2012, Istanbul Turkey, May 2012, pp. 2338–2342.

Proceedings from the 3rd European Symposium on Multimodal Communication, Dublin, September 17-18, 2015

Author:
Patrizia Paggio
Title:
Coordination of Head Movements and Speech in First Encounter Dialogues
Note: the following are taken directly from CrossRef
Citations:
No citations available at the moment


Responsible for this page: Peter Berkesand
Last updated: 2017-02-21