Control Engineering Practice 11 (2003) 57–66 Experimental design for the economic performance evaluation of industrial controllers I.K. Craig a, *, I. Koch b a Department of Electrical, Electronic and Computer Engineering, University of Pretoria, Pretoria, 0002, South Africa b School of Mathematical and Physical Sciences, University of Newcastle, Callaghan, NSW, Australia Received 18 February 2002; accepted 16 April 2002 Abstract Experimental design techniques are routinely used in the social and biological sciences to determine the relative performance of different factors. However, it appears that these techniques are not commonly used by the industrial control community. As a consequence, incorrect conclusions are often reached from inappropriately designed plant improvement trials. This presentation introduces experiment design concepts as part of a framework for determining the economic benefits of advanced industrial control projects. Using improved level control in mineral flotation as an example, common incorrect experiments are demonstrated and examples of correct experimental designs are given. The contention is that greater use of these tools by the industrial control community would significantly enhance the validity of comparative trials. r 2003 Elsevier Science Ltd. All rights reserved. Keywords: Experimental design; Economic performance evaluation; Industrial control; Mineral processing; Froth flotation; Process control 1. Introduction This paper is an attempt to increase the exposure of experimental design techniques in the industrial control community, and to also illustrate the benefits of using such techniques. It builds on Craig and Henning (2000) and uses level control in mineral froth flotation as the process for which an experiment is designed. The aim of economic performance evaluation is to produce quantitative economic benefit measures that are statistically significant (Craig & Henning, 2000). These measures are derived from plant data obtained from carefully designed experiments, as discussed in this paper. The experiments are designed to generate unbiased production data that capture the economic performance of the process when the relevant control system is in use. Experiments referred to here are especially useful in detecting relatively small process improvements, which are significant in monetary terms, against a background of relatively large plant variations. Many experiment design techniques are available in the statistical litera- ture (see, e.g. Federer, 1955; Finney, 1960; Box, Hunter, & Hunter, 1978; Montgomery, 1984; Robinson, 2000). These techniques, which are widely used in the biological and social sciences, are however not well known in the process industries. As a consequence, plant improvement trials are often inappropriately designed, take too long, and lead to either no conclusion where a useful conclusion could have been reached, or the wrong conclusion (Napier-Munn, 1995). One of the reasons for experiment design techniques not being well known in the process industries, is that the physical sciences are often thought of as exact, whereas the biological and social sciences are not. When performing industrial experiments however, there are often many variables that are not under the control of the experimenter, which can result in considerable variation from observation to observation on the same material. It is usually not feasible to take a large number of observations due to e.g. the process disruptions that can result. Experiment design techniques can be used to obtain unbiased estimates, from relatively few observa- tions, of true differences with a specific degree of precision. Randomisation, replication and blocking are the three basic principles of experiment design. Randomisation is *Corresponding author. Tel.: +27-12-420-2712; fax: +27-12-362- 5000. E-mail addresses: icraig@postino.up.ac.za (I.K. Craig), inge@frey.newcastle.edu.au (I. Koch). 0967-0661/03/$-see front matter r 2003 Elsevier Science Ltd. All rights reserved. PII:S0967-0661(02)00094-1