Vol.:(0123456789) Quality & Quantity (2020) 54:1299–1316 https://doi.org/10.1007/s11135-020-00986-8 1 3 A comparison of measures to validate scales in voting advice applications Bastiaan Bruinsma 1 Published online: 29 April 2020 © The Author(s) 2020 Abstract Voting advice applications (VAAs) are online tools providing voting advice to their users. This voting advice is based on the match between the answers of the user and the answers of several political parties to a common questionnaire on political attitudes. To visualize this match, VAAs use a wide array of visualisations, most popular of which are the two- dimensional political maps. These maps show the position of both the political parties and the user in the political landscape, allowing the user to understand both their own position and their relation to the political parties. To construct these maps, VAAs require scales that represent the main underlying dimensions of the political space. This makes the correct construction of these scales important if the VAA aims to provide accurate and helpful voting advice. This paper presents three criteria that assess if a VAA achieves this aim. To illustrate their usefulness, these three criteria—unidimensionality, reliability and qual- ity—are used to assess the scales in the cross-national EUVox VAA, a VAA designed for the European Parliament elections of 2014. Using techniques from Mokken scaling analy- sis and categorical principal component analysis to capture the metrics, I fnd that most scales show low unidimensionality and reliability. Moreover, even while designers can— and sometimes do—use certain techniques to improve their scales, these improvements are rarely enough to overcome all of the problems regarding unidimensionality, reliability and quality. This leaves certain problems for the designers of VAAs and designers of similar type online surveys. Keywords Voting advice applications · Scales · Data quality Electronic supplementary material The online version of this article (https://doi.org/10.1007/s1113 5-020-00986-8) contains supplementary material, which is available to authorized users. * Bastiaan Bruinsma bruinsma@soz.uni-frankfurt.de 1 Institute of Political Science, Goethe University Frankfurt am Main, Frankfurt am Main, Germany