The Rise of Interactional Multimodality in Human-Computer Interaction

Authors

  • Nils Klowait The Russian Presidential Academy of National Economy and Public Administration, The Moscow School of Social and Economic Sciences, Sber Gamification Lab https://orcid.org/0000-0002-7347-099X
  • Maria A. Erofeeva The Russian Presidential Academy of National Economy and Public Administration, The Moscow School of Social and Economic Sciences, Sber Gamification Lab https://orcid.org/0000-0002-0874-5272

DOI:

https://doi.org/10.14515/monitoring.2021.1.1793

Keywords:

human-computer interaction, embodied conversational agents, conversation analysis, multimodality, interactive resources

Abstract

The field of human-computer interaction (HCI) investigates the intersection between the design of devices and users. From an early focus on interaction modeling based on psychological experiments, the field has since experienced a shift towards the study of how actual humans interact with autonomous devices. The field became conductive to ethnographic, observational and videographic studies of human-device interaction. Conversation-analytic HCI became possible. That said, this new wave of researchers was never truly able to dethrone the psychological common sense of the field. With recent developments in both the technical-sensorial capabilities and outward actuational range of embodied virtual agents, the field of HCI has once again returned to the question of the sequential unfolding of the interaction between users and intelligent agents, and the multimodal interactional repertoire that is deployed throughout. This review will highlight the situational orientation of high-impact research in the field, and relate it to the cotemporaneous development of ethnomethodological and conversation analytic frameworks.

Acknowledgments. The article was prepared in the framework of a research grant funded by the Ministry of Science and Higher Education of the Russian Federation (grant ID: 075-15-2020-908). The article was prepared in cooperation with the Sber (ex. – Sberbank’s) Gamification Lab.

Author Biographies

Nils Klowait, The Russian Presidential Academy of National Economy and Public Administration, The Moscow School of Social and Economic Sciences, Sber Gamification Lab

  • The Russian Presidential Academy of National Economy and Public Administration, Moscow, Russia
    • Research Fellow at the Center for Innovative Social Research
  • The Moscow School of Social and Economic Sciences, Moscow, Russia
    • Senior Research Fellow at the International Center for Contemporary Sociological Theory
  • The Sber Gamification Lab, Moscow, Russia
    • Researcher

 

Maria A. Erofeeva, The Russian Presidential Academy of National Economy and Public Administration, The Moscow School of Social and Economic Sciences, Sber Gamification Lab

  • The Russian Presidential Academy of National Economy and Public Administration, Moscow, Russia
    • Cand. Sci. (Soc.), Researcher at the Center for Sociological Research
  • The Moscow School of Social and Economic Sciences, Moscow, Russia
    • Senior Research Fellow at the International Center for Contemporary Sociological Theory
  • The Sber Gamification Lab, Moscow, Russia
    • Researcher

Published

2021-03-04

Issue

Section

THEORY AND HISTORY