A Possible Error in CLSI Document EP5-A2 When Assigning a Critical Decision Value for an Outlier Robert Frenkel a,* and Ian Farrance b We refer to the document EP5-A2, Evaluation of Precision Performance of Quantitative Measurement Methods; Approved Guideline, Second Edition (1). Section 9.6 in this document is titled “Preliminary Precision Evaluation” and states, in part: At or near the end of the protocol familiarization period, an initial evaluation of repeatability should be conducted. Twenty aliquots of an appropriate test material (or a com- plete ‘batch’ if less than 20) should be assayed in sequence. Ideally, two or more concentration levels should be used. The standard deviation and coefficient of variation of the results should be calculated. Section 10.7 in the document is titled “Detection of Outliers” and states, in part: Assuming appropriate quality control procedures will be used during the experiment, a fairly weak (low power) test is suggested to detect gross outliers in the data. The outlier test is derived from the data collected during the prelimi- nary precision test. Data collected during each run of the precision evaluation experiment are in pairs (duplicates). The following test should be used: 1. If the absolute value of the difference between the repli- cates exceeds 5.5 times the standard deviation deter- mined in the preliminary precision test (see Section 9.6), the pair should be rejected. 2. If such an outlier is found, the cause of the problem should be investigated and the run repeated for that analyte. The value 5.5 is derived from the upper 99.9% value of the normalized range for the difference be- tween two observations. We contend that the value 5.5, stated in Section 10.7 as the critical ratio for the absolute difference between replicates and the standard deviation as determined in Section 9.6, is in error. For brevity, we refer to this ratio as the critical value. If we as- sume that the within-pair correlation, namely the mutual correlation between the 2 measurements that make up any pair, is negligible, the correct critical value is 4.653 and may be rounded to 4.7. If we assume, as is more plausible, that there is a moderate positive correlation between the paired values, the critical value is approximately (depend- ing on the degree of correlation) 3.3. All these fig- ures refer to the upper 99.9% value as indicated in Section 10.7. We assume that the sample of 20 component measurements as obtained using the procedure described in Section 9.6 are drawn from a popula- tion with a Gaussian density distribution having mean l and variance r 2 and that s 2 is an unbiased estimate of r 2 . Let s denote the square root of s 2 and, with 20 measurements, we assume that s is a good approximation to the standard deviation (SD) of the population. Let w 1 and w 2 be 2 meas- urements that make up a pair. Each has expecta- tion l and so the difference d ¼ w 1  w 2 has an expected value of zero. The SD s d of d is given by a Former affiliation: National Measurement Institute, West Lindfield, New South Wales, Australia; b Discipline of Laboratory Medicine, School of Health and Biomedical Sciences, RMIT University, Bundoora, Victoria, Australia. *Address correspondence to this author at: National Measurement Institute, West Lindfield, New South Wales, Australia. E-mail frenkelfamily@ hotmail.com Received March 15, 2021; accepted May 28, 2021. DOI: 10.1093/jalm/jfab077 V C American Association for Clinical Chemistry 2021. All rights reserved. For permissions, please email: journals.permissions@oup.com. ................................................................................................... March 2022 | 07:02 | 613–616 | JALM 613 OPINION Downloaded from https://academic.oup.com/jalm/article/7/2/613/6349186 by guest on 19 May 2023