On the accuracy of the telephone survey about Crimea: a posteriori error analysis

Abstract

Between March 14 and 16 (2014) the Public Opinion Foundation and the Russian Public Opinion Research Center conducted a large-scale telephone survey devoted to the attitudes of Russians towards the accession of Crimea. After the publication of results representing an unprecedented support for the presidential decision a number of accusations of political preconception and methodological incorrectness were addressed to both companies. The key objective of the article is to reveal possible errors reflecting the work of the interviewers. A random sample of 608 interviews conducted by the Public Opinion Foundation was generated for the data array; the interviews were further listened and analyzed. Audio transcripts were also checked up. The study revealed that the survey was the highest quality. Significant deficiencies were witnessed only in 5 % of interviews, and they do not affect the answers even though fixed; the final distributions of answers with or without errors do not differ much. Difficulties concerning the questions during the interview fit well into the everyday conversation. Additional questions and clarifications on answers do not go against the logic of the conversation due to compact survey with a clearly defined thematic structure. New technologies of data recording and the scale of the survey helped solving two basic methodological problems: firstly, to assess errors and risks that may be caused by unconscientious interviewers; secondly, to rate call centers based on the results of the work but not on the reporting documentation. The first conclusion contributes to the general theory of error; the second one may be useful for choosing the most reliable partners for those research teams that have just entered the market of field studies.