Biol. Cybern. 57, 341-347 (1987) Biological Cybernetics 9 Springer-Verlag 1987 Bootstrap Variance Estimators for the Parameters of Small-Sample Sensory-Performance Functions D. H. Foster i and W. F. Bischof 2 x Department of Communication and Neuroscience, University of Keele, Keele, Staffordshire ST5 5BG, England 2 Alberta Centre for Machine Intelligence and Robotics, University of Alberta, Edmonton T6G 2E9, Canada Abstract. The bootstrap method, due to Bradley Efron, is a powerful, general method for estimating a variance or standard deviation by repeatedly resam- pling the given set of experimental data. The method is applied here to the problem of estimating the standard deviation of the estimated midpoint and spread of a sensory-performance function based on data sets com- prising 15-25 trials. The performance of the bootstrap estimator was assessed in Monte Carlo studies against another general estimator obtained by the classical "combination-of-observations" or incremental method. The bootstrap method proved clearly su- perior to the incremental method, yielding much smaller percentage biases and much greater effici- encies. Its use in the analysis of sensory-performance data may be particularly appropriate when traditional asymptotic procedures, including the probit- transformation approach, become unreliable. 1 Introduction In the majority of sensory-performance measurements the typical finding is that the level of performance varies monotonically and nonlinearly with the level of the stimulus. In practice, the set of data relating stimulus level to performance level may be sum- marized by a single number, the critical level of the stimulus that yields a criterion level of performance. Thus when the response is discrete, referring say to the frequency with which a particular stimulus is detected, the critical level of the stimulus, in this case the threshold, may be defined as the level which corre- sponds to a detection frequency of 50 %. The perfor- mance function is often called the psychometric func- tion (see Fig. 1a). In some situations, it may be possible to replicate the experiment and obtain several estimates of a parameter such as the threshold so that a mean value may be calculated. The reliability of such a point estimate is then typically provided by the variance or standard deviation calculated from the set of indivi- dual estimates. But, in other situations, replication of the experiment may be impossible. Given a single set of performance data, an estimate of the standard devi- ation of a parameter estimate derived from the data set may then be essential in assessing the significance of the parameter estimate, either absolutely or in relation to parameter estimates derived from other distri- butions. Additionally, the estimate of standard devi- ation may have an importance in its own right, particularly when there may be sensory pathology (compare Patterson et al. 1980). When replication of the experiment is possible, estimates of the standard deviations of individual parameter estimates may still be useful in forming the best (minimum-variance) estimate of the mean or in assessing the contribution of potential outliers to the mean. Depending on the method used to fit a model sensory-performance curve to a single set of data, estimates of the variances of the parameters of the model may be derived by classical asymptotic theory. In particular, if ~ is the estimate of the parameter of interest, obtained as the solution of a maximum- likelihood equation, its estimated standard deviation SD is given by S"b = [ - l/(OZL/a T2)] ~/2, where L is the likelihood and the partial derivative is evaluated at T. It is this relationship that is used to estimate the variance of the midpoint ("ED50") and of the slope in the classical probit-transformation ap- proach to analysing psychometric functions (Finney 1964). Some computer software packages routinely produce estimates of the standard deviations of para- meter estimates derived from the Hessian matrix of 2nd-order partial derivatives of the function describ- ing the goodness of fit.