Modeling immersive media experiences by sensing impact on subjects

Eleni Kroupi, Philippe Hanhart, Jong Seok Lee, Martin Rerabek, Touradj Ebrahimi

Research output: Contribution to journalArticlepeer-review

14 Citations (Scopus)


As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14 % ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved when high beta EEG band is fused with peripheral features (0.17 % ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The results reveal that the right cortex is more involved when sensation of reality is low, and the left when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.

Original languageEnglish
Pages (from-to)12409-12429
Number of pages21
JournalMultimedia Tools and Applications
Issue number20
Publication statusPublished - 2016 Oct 1

Bibliographical note

Funding Information:
The research leading to these results has been performed in the framework of two Swiss National Foundation for Scientific Research (FN 200020-132673-1 and FN 200021-143696-1), FP7 EC EUROSTAR funded Project - Transcoders Of the Future TeleVision (TOFuTV), QoE-Net Initial Training Network (H2020-MSCA-ITN-2014), the Basic Science Research Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT and Future Planning (MSIP), Korea (2013R1A1A1007822), and the IT Consilience Creative Program funded by MSIP, Korea (IITP-2015-R0346-15-1008).

Publisher Copyright:
© 2015, Springer Science+Business Media New York.

All Science Journal Classification (ASJC) codes

  • Software
  • Media Technology
  • Hardware and Architecture
  • Computer Networks and Communications


Dive into the research topics of 'Modeling immersive media experiences by sensing impact on subjects'. Together they form a unique fingerprint.

Cite this