ELSEVIER Research Policy 25 (1996) 309-319 research policy Issues and perspectives on evaluating manufacturing modernization programs 1 Irwin Feller *, Amy Glasmeier, Melvin Mark Department of Economics, The Pennsylvania State University, N253 Burrowes, UniversityPark, PA 16802, USA Abstract The establishment of a national system of federal-state manufacturing modernization centers to serve small- and medium-sized manufacturing systems is a major experiment in the US. The system's rapid expansion has made it impossible to begin with an experimental design that would systematically permit tests of the variables and relationships likely to effect the long-term economic effectiveness of these centers or their political and financial stability. Relatedly, many existing evaluation studies and journalistic narratives are marred by serious analytical and empirical flaws. It is not too late to attempt to improve practice in future evaluations. Several areas of expanded evaluation are described, including theories of industrial restructuring, regional technology infrastructure, public management, financing, evaluation design, evaluability assessment, firm learning, and measurement of benefits and costs. In each case, evaluation is presented as a means of simultaneously yielding information relevant to formative and summative program-level decisions and to hypothesis testing. I. Introduction This article is written from a two-fold vantage point: first, from the recognition that significant improvements are both needed and feasible (al- beit at a cost) in the evaluation of federal and state manufacturing modernization programs; and second, from experiences in the difficulties of implementing formal evaluation designs, espe- cially those employing experimental and quasi-ex- perimental formats. From these vantage points we (1) propose an expanded framework for evalu- ating manufacturing modernization programs; (2) present a summary assessment and analysis of the current state of knowledge; and (3) outline our perspective on directions of future research. * Corresponding author. 1 This article is an expanded version of the paper, "Issues in the Evaluation of Technology Modernization Programs" by Irwin Feller, presented at the workshop on the Evaluation of Industrial Modernization Programs, Atlanta, Georgia, September 1-3, 1993. We acknowledge the constructive com- ments of two reviewers without implicating them in our per- spective. 2. The purposes of evaluation: an expanded per- spective Why conduct evaluations? The purposes of program evaluation, according to Goldenberg, are to learn about a program's operations and effects, to control the behavior of those responsible for program implementation, and to influence the 0048-7333/96/$15.00 © 1996 Elsevier Science B.V. All rights reserved SSDI 0048-7333(95)00834-9