RESEARCH COMMUNICATIONS CURRENT SCIENCE, VOL. 117, NO. 6, 25 SEPTEMBER 2019 1079 *e-mail: gangan_prathap@hotmail.com Construct validity maps and the NIRF 2019 ranking of colleges Gangan Prathap* A.P.J. Abdul Kalam Technological University, Thiruvananthapuram 695 016, India In this study, we prepare construct validity maps from the National Institutional Ranking Framework (NIRF) 2019 data for the top 100 colleges in India. Tamil Nadu, Delhi and Kerala together have a dispro- portionate 82% share of the top-ranking colleges in the country that participated in the 2019 exercise. The higher education system in India comprises about 52,000 units of assessment from universities, premier institutes of technology and colleges to stand-alone in- stitutions, and many participate in the NIRF exercise. The NIRF score is computed from five broad parame- ters, of which one is a peer review-based perception score for participating institutions. Using its teaching, learning and resources parameter as a proxy for teaching and learning resources input and its research and professional practices and graduation outcomes parameters as proxies for teaching and research out- puts or outcomes, we also compute a quality or excel- lence proxy and from this compute a second-order X- score. The three scores, NIRF, perception and X are used in the context of construct validity to construct two-dimensional maps to determine how the top col- leges are placed with respect to each other. A quantit- ative estimate is obtained using Peirce’s measure of predictive success to determine if the use of one con- struct measure to predict another is acceptable or not. In terms of the construct validity paradigm, we are able to recognize possible biases in the peer review perception scores and also recommend that the X- score, which is based on an input–output model, may give a better representation of reality. Keywords: Bibliometrics, construct validity, institu- tional ranking, research evaluation. AS reported in a survey 1 there are 864 universities, 40,026 colleges and 11,669 stand-alone institutions in India, making a total of more than 52,000 units of assessment. The higher education sector is so fragmented that few institutions have the critical mass to deliver high-end research along with high-quality education. Prathap 2 used bibliometric and economic data from the National Institutional Ranking Framework (NIRF) 2017 rankings to show that the two top-ranking colleges if only research excellence is considered are both from one state: Loyola College, Chennai and Bishop Heber College, Tiruchirappalli in Tamil Nadu. The NIRF 2019 results have now been released. Among the top 100 colleges in India, Tamil Nadu (35), Delhi (29) and Kerala (18) now account for 82% of this elite listing (in 2018 this was even higher at 85%). Among the more than 52,000 units of assessment in the higher education system in India, many leading and aspiring institutions now participate in the NIRF exercise. The NIRF score is computed from five broad parameters, of which one is a peer review- based perception score for participating institutions. Us- ing its teaching, learning and resources (TLR) parameter as a proxy for teaching and learning resources input, and its research and professional practices (RPC) and gradua- tion outcomes (GO) parameters as proxies for teaching and research outputs or outcomes, we also compute a quality or excellence proxy and from this compute a second-order X-score 2,3 . The three scores, NIRF, percep- tion and X are used in the context of construct validity (CV) to construct two-dimensional maps to determine how the top colleges are placed with respect to each other. The CV maps enable us to recognize possible biases in the peer review perception scores and also recommend that the X-score, which is based on an input–output mod- el, may give a better representation of reality. A quantita- tive estimate is obtained using Peirce’s measure of predictive success 4 to determine if the use of one con- struct measure to predict another is acceptable or not. The NIRF has just released its 2019 ranking of higher educational institutions across India (https://www. nirfindia.org/2019/Ranking2019.html ). It arrives at a single NIRF score using scores from five broad generic groups: TLR – Teaching, learning and resources; RPC – Research and professional practices; GO – Graduation outcomes; OI – Outreach and inclusivity and PR – Perception. This five-dimensional model is developed from sub-heads with weights assigned to each broad head, and also to the sub-heads within each head. For each sub- head, a score is generated using suitably proposed me- trics, and the sub-head scores are then added to obtain scores for each individual head. The overall score is computed based on the weights allotted to each head. This score can take a maximum value of 100. Thus, a multi-dimensional input and output problem is com- pressed into a single score and institutions, irrespective of size or resources, are finally rank-ordered based on these scores. How valid is this score? This is a difficult question to answer, as we are dealing with the issue of validity in a complex social system. There is indeed no such thing as an independent ground truth. Perhaps, the closest we have here is the perception score, which is an observed varia- ble and not a latent variable emerging from a mathematical model. In the NIRF operationalization, this is included as an input to get a final NIRF score, although with a very low weighting, i.e. 10%. From the CV point of view (an elaboration of which follows in the next section), it is more meaningful to use the perception score as a baseline with the NIRF score to get a better appreciation of the