Opinion polarity detection in Twitter data combining shrinkage regression and topic modeling

Hyui Geon Yoon, Hyungjun Kim, Chang Ouk Kim, Min Song

Research output: Contribution to journalArticlepeer-review

37 Citations (Scopus)


We propose a method to analyze public opinion about political issues online by automatically detecting polarity in Twitter data. Previous studies have focused on the polarity classification of individual tweets. However, to understand the direction of public opinion on a political issue, it is important to analyze the degree of polarity on the major topics at the center of the discussion in addition to the individual tweets. The first stage of the proposed method detects polarity in tweets using the Lasso and Ridge models of shrinkage regression. The models are beneficial in that the regression results provide sentiment scores for the terms that appear in tweets. The second stage identifies the major topics via a latent Dirichlet analysis (LDA) topic model and estimates the degree of polarity on the LDA topics using term sentiment scores. To the best of our knowledge, our study is the first to predict the polarities of public opinion on topics in this manner. We conducted an experiment on a mayoral election in Seoul, South Korea and compared the total detection accuracy of the regression models with five support vector machine (SVM) models with different numbers of input terms selected by a feature selection algorithm. The results indicated that the performance of the Ridge model was approximately 7% higher on average than that of the SVM models. Additionally, the degree of polarity on the LDA topics estimated using the proposed method was compared with actual public opinion responses. The results showed that the polarity detection accuracy of the Lasso model was 83%, indicating that the proposed method was valid in most cases.

Original languageEnglish
Pages (from-to)634-644
Number of pages11
JournalJournal of Informetrics
Issue number2
Publication statusPublished - 2016 May 1

Bibliographical note

Publisher Copyright:
© 2016 Elsevier Ltd.

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Library and Information Sciences


Dive into the research topics of 'Opinion polarity detection in Twitter data combining shrinkage regression and topic modeling'. Together they form a unique fingerprint.

Cite this