Research Article Received 15 September 2010, Revised 21 October 2011, Accepted 23 October 2011 Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/asmb.940 A case study to demonstrate a Pareto Frontier for selecting a best response surface design while simultaneously optimizing multiple criteria Lu Lu a , Christine M. Anderson-Cook a * † and Timothy J. Robinson b Experimenting with limited resources often means that we are trying to get more out of a single experiment and balance compet- ing goals. Selecting a best response surface design when simultaneously optimizing multiple criteria requires carefully choosing measures and scales of different design criteria and then balancing the trade-offs between the criteria. This paper illustrates a decision-making process using a Pareto frontier to identify good design candidates and a Utopia point approach for selection of an optimal design based on several competing criteria. The Pareto approach shows substantial improvement over the classic desirabil- ity function method by offering the user greater flexibility in quantifying the robustness of designs to weight specifications and the sensitivity of the solutions to different choices of weights, scales, and metrics. Graphical methods are used for summarizing and extracting useful information for improved decision-making. Copyright © 2012 John Wiley & Sons, Ltd. Keywords: balancing competing objectives; design optimality; desirability functions; design robustness; trade-offs between criteria 1. Introduction Designed experiments are often used for understanding the underlying mechanisms of processes and systems. Characteris- tics of a good design are outlined in [1, p. 282]. Many criteria have been developed for evaluating different characteristics of a design. Standard designs such as full and fractional factorial designs, central composite designs and Box–Behnken designs are known to exhibit many desirable design characteristics, but these designs often require specific sample sizes and regular shaped design regions. In situations with irregularly shaped design regions or when sample size constraints do not permit a standard design, computer-generated designs are commonly utilized. Standard computer generated designs require the user to specify a model, the design size and an optimality criterion that reflects the user’s main objective for the experiment. As an example, if the overall goal is to produce the model with the most precise regression parameter estimates, the D-criterion is the most common optimality criterion. Other criteria such as G-optimality and I-optimality criteria are often utilized when the user is primarily interested in precise predictions. Still other design criteria exist for situations when the user seeks to protect against model mis-specification. When good characteristics are desired for multiple aspects of design performance, a universally ‘best’ design that out- performs all others for all criteria of interest typically does not exist, because criteria are often in direct competition with one another. For example, a design that optimizes the precision of regression parameter estimates may be quite poor in terms of its ability to evaluate homogeneous error variance throughout the design region and evaluate lack-of’fit. A good design is one that produces reliable results under a wide variety of user-specified objectives. Selection of an optimal design therefore, requires carefully balancing competing objectives based on a thorough study of the trade-offs between them. Others in the literature have also sought to balance multiple objectives to achieve better designs. As early as Ref. [2] there has been discussion about the need to consider several aspects of a design when making a final selection. More recently, Jones and Nachtsheim [3], Jones et al. [4], and Bates et al. [5] all proposed methods for making designs more robust to uncertainty in model specification. Other approaches to design construction based on a balanced set of objectives include Refs. [6, 7]. a Los Alamos National Laboratory, Los Alamos, NM 87545, USA b University of Wyoming, Laramie, WY 82071, USA *Correspondence to: Christine M. Anderson-Cook, Los Alamos National Laboratory, Los Alamos, NM 87545, USA. † E-mail: c-and-cook@lanl.gov Copyright © 2012 John Wiley & Sons, Ltd. Appl. Stochastic Models Bus. Ind. 2012