Software Tools for Technology Transfer manuscript No. (will be inserted by the editor) LTL Satisfiability Checking ⋆ Kristin Y. Rozier 1 ⋆⋆ , Moshe Y. Vardi 2 1 NASA Ames Research Center, Moffett Field, California, 94035. ⋆⋆⋆ e-mail: Kristin.Y.Rozier@nasa.gov 2 Rice University, Houston, Texas 77005. e-mail: vardi@cs.rice.edu Abstract. We report here on an experimental investigation of LTL satisfiability checking via a reduction to model check- ing. By using large LTL formulas, we offer challenging model- checking benchmarks to both explicit and symbolic model checkers. For symbolic model checking, we use CadenceSMV, NuSMV, and SAL-SMC. For explicit model checking, we use SPIN as the search engine, and we test essentially all pub- licly available LTL translation tools. Our experiments result in two major findings. First, most LTL translation tools are re- search prototypes and cannot be considered industrial quality tools. Second, when it comes to LTL satisfiability checking, the symbolic approach is clearly superior to the explicit ap- proach. 1 Introduction Model-checking tools are successfully used for checking whether systems have desired properties [12]. The applica- tion of model-checking tools to complex systems involves a nontrivial step of creating a mathematical model of the sys- tem and translating the desired properties into a formal spec- ification. When the model does not satisfy the specification, model-checking tools accompany this negative answer with a counterexample, which points to an inconsistency between the system and the desired behaviors. It is often the case, how- ever, that there is an error in the system model or in the formal ⋆ An earlier version of this paper appeared in SPIN ’07. ⋆⋆ Work contributing to this paper was completed at Rice University, Cam- bridge University, and NASA, and was supported in part by the Rice Compu- tational Research Cluster (Ada), funded by NSF under Grant CNS-0421109 and a partnership between Rice University, AMD and Cray. ⋆⋆⋆ The use of names of software tools in this report is for accurate reporting and does not constitute an official endorsement, either expressed or implied, of such software by the National Aeronautics and Space Administration. specification. Such errors may not be detected when the an- swer of the model-checking tool is positive: while a positive answer does guarantee that the model satisfies the specifica- tion, the answer to the real question, namely, whether the sys- tem has the intended behavior, may be different. The realization of this unfortunate situation has led to the development of several sanity checks for formal verification [31]. The goal of these checks is to detect errors in the system model or the properties. Sanity checks in industrial tools are typically simple, ad hoc tests, such as checking for enabling conditions that are never enabled [33]. Vacuity detection pro- vides a more systematic approach. Intuitively, a specification is satisfied vacuously in a model if it is satisfied in some non-interesting way. For example, the linear temporal logic (LTL) specification (req → ♦grant) (“every request is even- tually followed by a grant”) is satisfied vacuously in a model with no requests. While vacuity checking cannot ensure that whenever a model satisfies a formula, the model is correct, it does identify certain positive results as vacuous, increasing the likelihood of capturing modeling and specification errors. Several papers on vacuity checking have been published over the last few years [2, 3, 9, 29, 28, 32, 36, 39], and various indus- trial model-checking tools support vacuity checking [2,3,9]. All vacuity-checking algorithms check whether a subfor- mula of the specification does not affect the satisfaction of the specification in the model. In the example above, the sub- formula req does not affect satisfaction in a model with no requests. There is, however, a possibility of a vacuous result that is not captured by current vacuity-checking approaches. If the specification is valid, that is, true in all models, then model checking this specification always results in a positive answer. Consider for example the specification (b 1 → ♦b 2 ), where b 1 and b 2 are propositional formulas. If b 1 and b 2 are logically equivalent, then this specification is valid and is sat- isfied by all models. Nevertheless, current vacuity-checking approaches do not catch this problem. We propose a method for an additional sanity check to catch exactly this sort of oversight. Writing formal specifications is a difficult task, which is prone to error just as implementation development is error prone. However, formal verification tools offer little help in debugging specifications other than standard vacuity check- ing. Clearly, if a formal property is valid, then this is certainly due to an error. Similarly, if a formal property is unsatisfiable, that is, true in no model, then this is also certainly due to an error. Even if each individual property written by the speci- fier is satisfiable, their conjunction may very well be unsatis- fiable. Recall that a logical formula ϕ is valid iff its negation ¬ϕ is not satisfiable. Thus, as a necessary sanity check for de- bugging a specification, model-checking tools should ensure that both the specification ϕ and its negation ¬ϕ are satisfi- able. (For a different approach to debugging specifications, see [1].) A basic observation underlying our work is that LTL sat- isfiability checking can be reduced to model checking. Con- sider a formula ϕ over a set Prop of atomic propositions. If a model M is universal, that is, it contains all possible traces