Multimodal and Multi-Lingual Deep Neural Network for Interactive Behavior Style Recognition from Uncontrolled Video-logs of Children with Autism

Zhenhao Zhao, Eunsun Chung, Myungeun Lee, Kyong Mee Chung, Chung Hyuk Park

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

With the increase of prevalence in autism, the need for efficient public health support has been amplified. Socially-assistive robots (SARs) have been found effective in engaging and interacting with autistic children, however, the perception intelligence during interaction still needs more domain-specific knowledge in terms of understanding children's behaviors. The Family Observation Schedule-Second Version (FOS-II) is one of the key methods in assessing parent-child interactions in developmental disabilities, yet its manual annotation demands considerable time and effort. This study proposes a multimodal artificial intelligence (AI) model using video and audio inputs for automated FOS-II annotation. Utilizing advanced deep learning for behavior recognition, this method offers rapid, cost-effective FOS-II scaling. It will thus enhance the capability of socially assistive robots to understand human behaviors and support the advancement of digital health research for children with autism. The visual perception in home settings are most likely based on uncontrolled environments, so it is crucial to develop algorithms that can robustly work with video-log data with uncontrolled quality. Ultimately, it aims to ease the burden on parents and caregivers, streamlining the monitoring and treatment of challenging behaviors in autism.

Original languageEnglish
Title of host publication33rd IEEE International Conference on Robot and Human Interactive Communication, ROMAN 2024
PublisherIEEE Computer Society
Pages1644-1650
Number of pages7
ISBN (Electronic)9798350375022
DOIs
Publication statusPublished - 2024
Event33rd IEEE International Conference on Robot and Human Interactive Communication, ROMAN 2024 - Pasadena, United States
Duration: 2024 Aug 262024 Aug 30

Publication series

NameIEEE International Workshop on Robot and Human Communication, RO-MAN
ISSN (Print)1944-9445
ISSN (Electronic)1944-9437

Conference

Conference33rd IEEE International Conference on Robot and Human Interactive Communication, ROMAN 2024
Country/TerritoryUnited States
CityPasadena
Period24/8/2624/8/30

Bibliographical note

Publisher Copyright:
© 2024 IEEE.

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software

Fingerprint

Dive into the research topics of 'Multimodal and Multi-Lingual Deep Neural Network for Interactive Behavior Style Recognition from Uncontrolled Video-logs of Children with Autism'. Together they form a unique fingerprint.

Cite this