Effects of questionnaire design on data quality in student online surveys

Abstract

The authors tried to define the elements of the optimal questionnaire design in student opinion surveys. Four full factorial experiments were carried out. The first one aimed at revealing the most suitable format of providing the answers in percentages. Respondents were randomly offered for one out of two experimental question formats: an input box with values 0-100 and a slider. The second one was needed to compare evaluative questions in grid format and button format. The third experiment allowed studying the effects of the simultaneous (single screen) and gradual (different screens) questioning measuring different aspects of the same phenomenon on the respondent answers. The fourth experiment was made to examine the length of the answers to the open-ended questions depending on the size of the input screen. The results revealed that (1) the input screen question format leads to the increased number of the undecided compared to the slider format; (2) grid-format of evaluation questions decreases the time spent on answers but increases the unfair answer rate; (3) providing several questions referring to different aspects of the same issue on the same screen allows getting more concerted responses; (4) the size of the comments screen does not have any statistical effect on the desire of respondents to leave comments as well as on the length of the comment.