New computer-intensive procedures for testing null hypotheses comparing two parameters approximately Kousuke Okamoto a , Makiko Higashi b , Masahiko Yokota a , Rika Nishikiori b , Mie Osaki b , Teruo Yasunaga c , Tatsuya Takagi a,c, * a Graduate School of Pharmaceutical Sciences, Osaka University, 1-6 Yamadaoka, Suita, Osaka 565-0871 Japan b Faculty of Pharmaceutical Sciences, Osaka University, 1-6 Yamadaoka, Suita, Osaka 565-0871, Japan c Genome Information Research Center, Osaka University, 3-1 Yamadaoka, Suita, Osaka 565-0871 Japan Received 28 November 2004; received in revised form 1 April 2005; accepted 11 April 2005 Available online 27 September 2005 Abstract For a long time, statistical tests of significance have tested a null hypothesis of the form, H 0 : l 1 = l 2 . However, in many cases, it is more important whether a null hypothesis of the form, H 0 : l 1 å l 2 , is rejected or not. When the former hypothesis is judged, no null hypotheses are accepted if the sample size of a data set is sufficiently large. In order to avoid this problem, and to judge instead the latter hypothesis, a fixed D test has been often used. However, the fixed D test has some problems. For example, the fixed D test is only appropriate for testing averages or ratios. In addition, the test requires D to be specified in advance, even if the tester must specify D subjectively. Thus, more objective procedures for judging the null hypothesis, H 0 : l 1 å l 2 , are required. In this study, we suggest new procedures which enable us to judge the null hypothesis, H 0 : l 1 å l 2 , without specifying D in advance using a re-sampling method. Our new procedures are widely applicable to various statistics, and enable us to obtain confidence intervals of confidence intervals. Moreover, by the application of these new procedures to simulation trials, we further demonstrate that the procedures have sufficient statistical power. D 2005 Elsevier B.V. All rights reserved. Keywords: Re-sampling; Computer-intensive statistical method; Statistical test; Fuzzy; Null hypothesis 1. Introduction When applying statistical tests of significance, a null hypothesis about one parameter is proposed, tested and judged, and, on the basis of the results, either rejected or accepted. In most cases, a null hypothesis is formulated in terms of whether the parameter is equal to a constant or another parameter, such as H 0 : l 1 = l 2 . If the null hypothesis is rejected, the alternative hypothesis H 1 : l 1 m l 2 (H 1 : l 1 > l 2 or H 1 : l 1 < l 2 ) is accepted. When applying statistical tests of significance, practitioners are usually seeking to determine if two parameters are different, or whether one parameter is superior to another. If the difference between two parameters is significant according to the statistical test, the practitioners tend to consider that the difference may also be considered significant for the purposes of practical judgments. However, even if the difference is significant, the difference between two parameters may not be significant for practical and implicit judgments. For example, when one average is 25.0 and another is 25.1, the difference between the two averages is probably not significant for practical judgments. However, if the data set is large enough, the difference will always become statistically significant. Ordinal statistical tests always judge a null hypothesis of the form that one parameter is strictly equal to another. However, for practical and natural judgments, the difference between two parameters is judged approximately. Thus there is a discrepancy concerning the significance as described above. If a null hypothesis of the form that one parameter is nearly equal to another, such as H 0 : l 1 å l 2 , can be statistically judged, this discrepancy will be diminished. The statistical judgment of the null hypothesis, H 0 : l 1 å l 2 , is closer to natural, practical judgments than that of the null hypothesis, H 0 : l 1 = l 2 . 0169-7439/$ - see front matter D 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.chemolab.2005.04.014 * Corresponding author. Graduate School of Pharmaceutical Sciences, Osaka University, 1-6 Yamadaoka, Suita, Osaka 565-0871 Japan. Tel.: +81 6 6879 8254; fax: +81 6 6879 8250. E-mail address: ttakagi@phs.osaka-u.ac.jp (T. Takagi). Chemometrics and Intelligent Laboratory Systems 82 (2006) 66 – 74 www.elsevier.com/locate/chemolab