Techniques For Safety Critical Software Development James D. Kiper Department of Systems Analysis Miami University Oxford, OH 450565 kiperjd@muohio.edu James E. Tomayko School of Computer Science Carnegie Mellon University Pittsburgh PA 15213 jet@cs.cmu.edu Although the dreams of artificial intelligence have not been realized (despite the recent success of IBM in chess), it is apparent that software control of machines is quickly permeating our lives in many ways – some seen, some hidden. NASA is clearly dependent on computers to fly the space shuttle or to run the planned space station. Many commercial and military aircraft alike require computer systems to maintain flight and navigate. Power plants that produce electricity from nuclear fuel require software to monitor and control system conditions. Robot control of manufacturing machines and assembly lines implies embedded software. These and other application area require software systems that not only work correctly, but also satisfy safety concerns. A growing number of software systems can now be labeled safety critical. Peter Neumann writes [9, p.3]: Increasingly, we depend on computer systems to behave acceptably in applications with extremely critical requirements, by which we mean that the failure of systems to meet their requirements may result in serious consequences...We explore various inherent limitations both of the technology and of the people who interact with it. Certain limitations can be overcome – albeit only with significant effort. We must strive to promote the development and systematic use of techniques that can help us to identify the intrinsic limitations, and to reduce those that are not intrinsic—for example, through better systems and better operational practices. Examples of catastrophic failures of software systems are not difficult to find. The column “Risks to the Public” [11] in Software Engineering Notes describes many specific examples of risks to the public caused by software failures. The Therac-25 accidents [7] are some of the most widely reported examples of computer system failure. In the software engineering field, there is a wide recognition of the importance of accurate requirements. This is most apparent among researchers in formal methods of specification. It is obvious that a safe system will generally not result from inaccurate, incomplete, or inconsistent specifications. The last decade has seen increased study and use of formal specification languages like Z [13], VDM [4], and Larch. [6] Formal specification provide a vehicle for achieving specifications that can be proven to be consistent and complete. Hazard analysis has become an important part of requirements analysis. [8] However, formal specification involves more than a formal specification language. Wing [14] makes this point. Since a formal method is a method and not just a computer program or language, it may or may not have tool support. If the syntax of a formal method’s specification language is made explicit, providing standard syntax analysis tools for formal specifications would be appropriate. If the language’s semantics are sufficiently restricted, varying degrees of semantic analysis can be performed with machine aids as well. Thus, formal specifications have the additional advantage over informal ones of being amenable to machine analysis and manipulation. The value of formal specification is enhanced when coupled with formal verification. A formal verification technique ascertains the degree to which results of each development phase meet the specification from the previous phase. There are a range of verification techniques that can be considered formal: statistical testing [2], code and document inspection [5], and formal verification of design and code. [12] Theorem provers like the one in the tool PVS [1] allow properties of specification to be proven (or disproved). There have been many critics of the use of formal methods. Many practitioners have questioned the added value of formal methods, have asserted that typical software engineers are incapable of the required mathematics, and have debated whether the additional time spent has sufficient benefit. Hall addresses some of these “myths of formal methods” in [3]. Peter Neumann cites several success in the use of formal methods in [10]. John Rushby [12, pp. 12-13] adds this perspective on the value of the use of formal methods: There are advantages, difficulties, and costs associated with the use of formal methods. The balance of advantage varies with the nature and 1060-3425/98 $10.00 (C) 1998 IEEE