Mining opinions on the Internet: can the text analysis methods replace public opinion polls?

Abstract

User-generated content becomes an alternative or supplementary source for public opinion studies. Nevertheless, the question whether modern methods of data processing can completely or partially replace opinion surveys remains unclear. The purpose of this article is to show methodological possibilities of topic modeling and opinion mining based on large text dataset analysis. The article provides a comparison between the analysis of YouTube user comments and the results of the public opinion survey devoted to the Chaika documentary produced by the Anti-Corruption Foundation. The study reveals that the analysis of opinions collected from the Internet cannot completely replace opinion surveys; however, it provides a broader context for interpreting the opinions and their more detailed assessments. Additionally, it can be used to improve the questionnaire structure and the quality of questions.