Digital Imaging for Online Monitoring and Control of Industrial Snack Food Processes Honglu Yu and John F. MacGregor* McMaster Advanced Control Consortium, Department of Chemical Engineering, McMaster University, Hamilton, Ontario L8S 4L7, Canada Gabe Haarsma † and Wilfred Bourg Frito-Lay Technology R&D, Frito-Lay, Inc., P.O. Box 660634, Dallas, Texas 75266-0634 Results from the implementation of an online color imaging system on industrial snack food production lines are presented. Feature information is extracted from images using multivariate image analysis based on principal component analysis and is used to develop models to predict the coating content and the coating distribution on the products. The imaging system is used to monitor these product quality variables and to detect and diagnose operational problems in the plants. It is also used to directly implement closed-loop feedback control over the coating concentration. 1. Introduction The availability of informative, inexpensive, and robust online sensors is one of the most important factors for the successful monitoring and control of processes. The petrochemical industry made rapid advances in multivariable model predictive control largely because they had the availability and abundance of inexpensive and informative sensors such as ther- mocouples, pressure transducers, flowmeters, pH and ion-specific meters, and gas chromatographs. This is a direct result of the fact that the major streams in petrochemical processes consist of well-mixed gases and liquids, which made the use of such sensors very easy. On the other hand, the solids processing industry has had much less success at implementing advanced control precisely because of the lack of such sensors. However, with the advent of inexpensive digital cameras over the past decade, things are changing rapidly. Today an RGB (red, green, and blue) color camera connected to a fairly powerful PC is on the order of only a few thousand dollars or so. In contrast, to insert a simple thermocouple well into a process line or a reactor is considerably more expensive. If affordable digital imag- ing systems can be used to effectively extract subtle information on the behavior of a process or on the quality of the product, then it could indeed lead to a more rapid application of advanced control in process industries manufacturing of solid products such as pulp and paper, polymer sheet and films, and food products. In this paper we report on the development of such an online imaging system and its use for the online monitoring and feedback control of product quality variables in the snack food industry. Much of the literature on digital image processing involves methods for altering the visual image in some way in order to make it more visually appealing or to extract information on the shapes, boundaries, or loca- tion of various observable features. In this sense, traditional image processing techniques 1-3 serve as automated vision systems performing operations faster and more precisely than human operators. These are indeed a very important class of problems. However, many quality monitoring and control problems are more similar to those treated in this paper. They do not involve image enhancement issues but rather the extraction of subtle information from the image (much of which is not readily visible to the human eye) that is related to product quality. For example, in this paper we are concerned with the prediction of the average coating concentrations and the distribution of the coat- ing on snack food products passing on a moving belt under the imaging system. In these situations, image processing is not concerned with image enhancement or even with the image space at all. Rather, the problem is one of information extraction from the image and the use of such information for prediction, monitoring, and control. For this purpose a different set of techniques falling under the heading of multivariate image analysis (MIA), 4-6 which employs multivariate statistical tech- niques such as principal component analysis (PCA) and partial least squares (PLS), have been developed. In this approach, most of the analysis is done in the latent variable feature space rather than in the image space. Although most of the MIA methods have been applied to the analysis of single still images, an indication of their potential for monitoring time-varying images was presented by Bharati and MacGregor 7 and subsequently applied to the online monitoring of lumber defects 8 and pulp and paper quality. 8 In Yu and MacGregor, 9 several MIA and multivariate image regression techniques for the extraction of the coating content and distribution from time-varying images of snack food products were developed. The most robust of those methods is used in this paper for the online monitoring and control of these snack food product lines. The paper is organized as follows. Following some general background on digital images and on MIA, an overview of the methodology used for the prediction of * To whom correspondence should be addressed. Tel.: (905)- 525-9140 ext. 24951. Fax: (905)521-1350. E-mail: macgreg@ mcmaster.ca. † Present address: Westhollow Technology Center, Shell Global Solutions (U.S.) Inc., Houston, TX 77082-3102. 3036 Ind. Eng. Chem. Res. 2003, 42, 3036-3044 10.1021/ie020941f CCC: $25.00 © 2003 American Chemical Society Published on Web 05/20/2003