Affect recognition based on physiological changes during the watching of music videos

Ashkan Yazdani, Jong Seok Lee, Jean Marc Vesin, Touradj Ebrahimi

Research output: Contribution to journalArticlepeer-review

37 Citations (Scopus)


Assessing emotional states of users evoked during their multimedia consumption has received a great deal of attention with recent advances in multimedia content distribution technologies and increasing interest in personalized content delivery. Physiological signals such as the electroencephalogram (EEG) and peripheral physiological signals have been less considered for emotion recognition in comparison to other modalities such as facial expression and speech, although they have a potential interest as alternative or supplementary channels. This article presents our work on: (1) constructing a dataset containing EEG and peripheral physiological signals acquired during presentation of music video clips, which ismade publicly available, and (2) conducting binary classification of induced positive/negative valence, high/low arousal, and like/dislike by using the aforementioned signals. The procedure for the dataset acquisition, including stimuli selection, signal acquisition, self-assessment, and signal processing is described in detail. Especially, we propose a novel asymmetry index based on relative wavelet entropy for measuring the asymmetry in the energy distribution of EEG signals, which is used for EEG feature extraction. Then, the classification systems based on EEG and peripheral physiological signals are presented. Single-trial and single-run classification results indicate that, on average, the performance of the EEG-based classification outperforms that of the peripheral physiological signals. However, the peripheral physiological signals can be considered as a good alternative to EEG signals in the case of assessing a user's preference for a given music video clip (like/dislike) since they have a comparable performance to EEG signals while being more easily measured.

Original languageEnglish
Pages (from-to)1-26
Number of pages26
JournalACM Transactions on Interactive Intelligent Systems
Issue number1
Publication statusPublished - 2012 Mar

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Artificial Intelligence


Dive into the research topics of 'Affect recognition based on physiological changes during the watching of music videos'. Together they form a unique fingerprint.

Cite this