What emotions make one or five stars? Understanding ratings of online product reviews by sentiment analysis and xai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

When people buy products online, they primarily base their decisions on the recommendations of others given in online reviews. The current work analyzed these online reviews by sentiment analysis and used the extracted sentiments as features to predict the product ratings by several machine learning algorithms. These predictions were disentangled by various methods of explainable AI (XAI) to understand whether the model showed any bias during prediction. Study 1 benchmarked these algorithms (knn, support vector machines, random forests, gradient boosting machines, XGBoost) and identified random forests and XGBoost as best algorithms for predicting the product ratings. In Study 2, the analysis of global feature importance identified the sentiment joy and the emotional valence negative as most predictive features. Two XAI visualization methods, local feature attributions and partial dependency plots, revealed several incorrect prediction mechanisms on the instance-level. Performing the benchmarking as classification, Study 3 identified a high no-information rate of 64.4% that indicated high class imbalance as underlying reason for the identified problems. In conclusion, good performance by machine learning algorithms must be taken with caution because the dataset, as encountered in this work, could be biased towards certain predictions. This work demonstrates how XAI methods reveal such prediction bias.

Original languageEnglish
Title of host publicationArtificial Intelligence in HCI - 1st International Conference, AI-HCI 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Proceedings
EditorsHelmut Degen, Lauren Reinerman-Jones
PublisherSpringer
Pages412-421
Number of pages10
ISBN (Print)9783030503338
DOIs
Publication statusPublished - 2020
Event1st International Conference on Artificial Intelligence in HCI, AI-HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 - Copenhagen, Denmark
Duration: 2020 Jul 192020 Jul 24

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12217 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference1st International Conference on Artificial Intelligence in HCI, AI-HCI 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020
Country/TerritoryDenmark
CityCopenhagen
Period20/7/1920/7/24

Bibliographical note

Publisher Copyright:
© Springer Nature Switzerland AG 2020.

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • General Computer Science

Fingerprint

Dive into the research topics of 'What emotions make one or five stars? Understanding ratings of online product reviews by sentiment analysis and xai'. Together they form a unique fingerprint.

Cite this