Enhancing Lexicon-Based Review Classification by Merging and Revising Sentiment Dictionaries

This paper presents a method of improving lexicon-based review classification by merging multiple sentiment dictionaries, and selectively removing and switching the contents of merged dictionaries. First, we compare the positive/negative book review classification performance of eight individual sentiment dictionaries. Then, we select the seven dictionaries with greater than 50% accuracy and combine their results using (1) averaging, (2) weighted-averaging, and (3) majority voting. We show that the combined dictionaries perform only slightly better than the best single dictionary (65.8%) achieving (1) 67.8%, (2) 67.7%, and (3) 68.3% respectively. To improve this, we combine seven dictionaries at a deeper level by merging the dictionary entry words and averaging the sentiment scores. Moreover, we leverage the skewed distribution of positive/negative threshold setting data to update the merged dictionary by selectively removing the dictionary entries that do not contribute to classification while switching the polarity of selected sentiment scores that hurts the classification performance. We show that the revised dictionary achieves 80.9% accuracy and outperforms both the individual dictionaries and the shallow dictionary combinations in the book review classification task.