DEAP: A database for emotion analysis; Using physiological signals

Sander Koelstra; Christian Mühl; Mohammad Soleymani; Jong Seok Lee; Ashkan Yazdani; Touradj Ebrahimi; Thierry Pun; Anton Nijholt; Ioannis Patras

doi:10.1109/T-AFFC.2011.15

DEAP: A database for emotion analysis; Using physiological signals

Sander Koelstra, Christian Mühl, Mohammad Soleymani, Jong Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, Ioannis Patras

Research output: Contribution to journal › Article › peer-review

3651 Citations (Scopus)

Abstract

We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.

Original language	English
Article number	5871728
Pages (from-to)	18-31
Number of pages	14
Journal	IEEE Transactions on Affective Computing
Volume	3
Issue number	1
DOIs	https://doi.org/10.1109/T-AFFC.2011.15
Publication status	Published - 2012 Jan

Bibliographical note

Funding Information:
The research leading to these results has been performed in the frameworks of European Community’s 17th Framework Program (FP7/2007-2011) under grant agreement no. 216444 (PetaMedia). Furthermore, the authors gratefully acknowledge the support of the BrainGain Smart Mix Programme of the Netherlands Ministry of Economic Affairs, the Netherlands Ministry of Education, Culture, and Science and the Swiss National Foundation for Scientific Research and the NCCR Interactive Multimodal Information Management (IM2). The authors also thank Sebastian Schmiedeke and Pascal Kelm at the Technische Universität Berlin for performing the shot boundary detection on this data set.

All Science Journal Classification (ASJC) codes

Software
Human-Computer Interaction

Access to Document

10.1109/T-AFFC.2011.15

Cite this

@article{aa3a21c929294e6fadf071ffca1e1421,

title = "DEAP: A database for emotion analysis; Using physiological signals",

abstract = "We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.",

keywords = "EEG, Emotion classification, affective computing, pattern classification, physiological signals, signal processing",

author = "Sander Koelstra and Christian M{\"u}hl and Mohammad Soleymani and Lee, {Jong Seok} and Ashkan Yazdani and Touradj Ebrahimi and Thierry Pun and Anton Nijholt and Ioannis Patras",

note = "Funding Information: The research leading to these results has been performed in the frameworks of European Community{\textquoteright}s 17th Framework Program (FP7/2007-2011) under grant agreement no. 216444 (PetaMedia). Furthermore, the authors gratefully acknowledge the support of the BrainGain Smart Mix Programme of the Netherlands Ministry of Economic Affairs, the Netherlands Ministry of Education, Culture, and Science and the Swiss National Foundation for Scientific Research and the NCCR Interactive Multimodal Information Management (IM2). The authors also thank Sebastian Schmiedeke and Pascal Kelm at the Technische Universit{\"a}t Berlin for performing the shot boundary detection on this data set.",

year = "2012",

month = jan,

doi = "10.1109/T-AFFC.2011.15",

language = "English",

volume = "3",

pages = "18--31",

journal = "IEEE Transactions on Affective Computing",

issn = "1949-3045",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "1",

}

TY - JOUR

T1 - DEAP

T2 - A database for emotion analysis; Using physiological signals

AU - Koelstra, Sander

AU - Mühl, Christian

AU - Soleymani, Mohammad

AU - Lee, Jong Seok

AU - Yazdani, Ashkan

AU - Ebrahimi, Touradj

AU - Pun, Thierry

AU - Nijholt, Anton

AU - Patras, Ioannis

N1 - Funding Information: The research leading to these results has been performed in the frameworks of European Community’s 17th Framework Program (FP7/2007-2011) under grant agreement no. 216444 (PetaMedia). Furthermore, the authors gratefully acknowledge the support of the BrainGain Smart Mix Programme of the Netherlands Ministry of Economic Affairs, the Netherlands Ministry of Education, Culture, and Science and the Swiss National Foundation for Scientific Research and the NCCR Interactive Multimodal Information Management (IM2). The authors also thank Sebastian Schmiedeke and Pascal Kelm at the Technische Universität Berlin for performing the shot boundary detection on this data set.

PY - 2012/1

Y1 - 2012/1

N2 - We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.

AB - We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.

KW - EEG

KW - Emotion classification

KW - affective computing

KW - pattern classification

KW - physiological signals

KW - signal processing

UR - http://www.scopus.com/inward/record.url?scp=84859916185&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859916185&partnerID=8YFLogxK

U2 - 10.1109/T-AFFC.2011.15

DO - 10.1109/T-AFFC.2011.15

M3 - Article

AN - SCOPUS:84859916185

SN - 1949-3045

VL - 3

SP - 18

EP - 31

JO - IEEE Transactions on Affective Computing

JF - IEEE Transactions on Affective Computing

IS - 1

M1 - 5871728

ER -

DEAP: A database for emotion analysis; Using physiological signals

Abstract

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this