Personality assessment, ‘construct validity’, and the significance of theory q Simon Boag Department of Psychology, Macquarie University, Sydney, NSW 2109, Australia article info Article history: Received 12 September 2014 Received in revised form 16 December 2014 Accepted 22 December 2014 Available online xxxx Keywords: Latent variables Likert scales Measurement Personality assessment Personality traits Psychometrics Realism Theoretical variables abstract Personality assessment helps us to predict how people behave under various circumstances or how well a person might perform within certain roles. However, there are reasons to question the supposed ‘con- struct validity’ of tests designed to assess various personality attributes including dispositional traits. To demonstrate this, the paper first discusses a realist account of test validity where validity requires that both the attribute exist and that changes in the attribute are causally related to changes in test scores. The paper demonstrates that the validity for tests of dispositional traits is questionable given conceptual problems with traits existing as within-person attributes capable of causing changes in test scores. The widespread reliance on Likert-style response formats is then discussed in relation to the assumed quan- titative structure of personality attributes. Based on a realist view of measurement, the uncritical adop- tion of a representational theory of measurement within personality research means that the validity of all personality tests claiming to ‘measure’ personality attributes is questionable. Suggestions for address- ing test validity in personality assessment are then discussed in terms of paying greater critical attention to personality theory itself and adopting a realist theory of assessment and measurement. Ó 2014 Elsevier Ltd. All rights reserved. 1. Introduction Personality assessment currently helps predict how people behave under various circumstances or how well a person might perform within certain roles (e.g., Barrick, Mount, & Judge, 2001; Berry, Ones, & Sackett, 2007; Furnham & Fudge, 2008; Salgado, 2003). However, what personality tests are actually assessing is not entirely clear and whether personality variables such as dispo- sitional traits are even sensibly formulated remains a matter of dis- pute. The specific issue here concerns the clarity of personality attributes such as dispositional traits as found in the Five Factor Theory (FFT) and Model (FFM) (Costa & McCrae, 1995, 2008, 2009). Long-standing arguments propose that dispositional traits are descriptive summaries reified into within-subject attributes and then erroneously used to explain individual behaviour (Boag, 2011a, 2011b; Cervone, 1999, 2004, 2005; Kroger & Wood, 1993). Consequently, whether personality assessment procedures assess what they purport to assess is questionable because the under- standing of the personality attribute itself is confused. The aim of this paper is to address the issue of validity in per- sonality assessment procedures and to demonstrate the signifi- cance of theory for developing valid personality tests. The paper first discusses Borsboom, Mellenbergh, and van Heerden’s (2004) realist account of test validity. This realist position is applicable to any personality tests claiming to assess actual personality attri- butes, including tests of dispositional traits, where traits are hypothesised to be attributes that are causally reflected in person- ality test scores (e.g., Costa & McCrae, 1995, 2008, 2009). While there are limitations with Borsboom et al.’s (2004) account with respect to assuming that attributes are quantitative (Michell, 2009b), a realist account of test validity provides two criteria for valid assessment: (i) the attribute in question exists, and; (ii) that differences in the attribute are causally reflected in differences in scores upon the assessment outcomes. As will be developed, the immediate problem with the assessment of dispositional traits is that neither (i) nor (ii) are satisfied: the reality of dispositional traits is questionable given logical problems in their conceptualisa- tion. Specifically, dispositional trait accounts appear to reify person–situation relationships into attributes possessed by indi- viduals and the subsequent problem of circularity precludes any causal influence of traits upon test performance (Boag, 2011a, 2011b). The paper then turns its attention to the broader issue of http://dx.doi.org/10.1016/j.paid.2014.12.039 0191-8869/Ó 2014 Elsevier Ltd. All rights reserved. q An earlier version of the paper was presented at the 13th Australian Conference on Personality and Individual Differences (ACPID) hosted by Newcastle University, November 28–29, 2014. I would like to thank three anonymous reviewers for their valuable comments. E-mail address: simon.boag@mq.edu.au URL: http://www.simonboag.com Personality and Individual Differences xxx (2015) xxx–xxx Contents lists available at ScienceDirect Personality and Individual Differences journal homepage: www.elsevier.com/locate/paid Please cite this article in press as: Boag, S. Personality assessment, ‘construct validity’, and the significance of theory. Personality and Individual Differences (2015), http://dx.doi.org/10.1016/j.paid.2014.12.039