Percentage-based versus SAFE Vote Tabulation Auditing: A Graphic Comparison 29 June 2007 DRAFT version 1 DRAFT 29 June 2007 JMc/ML Percentage-based versus SAFE Vote Tabulation Auditing: A Graphic Comparison John McCarthy * , Howard Stanislevic**, Mark Lindeman***, Arlene Ash****, Vittorio Addona*****, Mary Batcher****** Abstract Trustworthy elections require comprehensive auditing and corrective action to eliminate major errors in counting votes. In this paper, we address just one component of electoral audits: specifying how many randomly selected precincts should undergo hand recounts to decide whether the winner in the electronic tally should be confirmed. Several pending electoral-integrity bills specify hand audits of between 2 and 10% of all precincts. However, percentage-based audits are usually inefficient, because they use large samples for large jurisdictions, even though the sample needed to achieve good accuracy is much more affected by the closeness of the race than the size of the population. Percentage-based audits can also be ineffective, since close races may require auditing a large fraction of the total – even a 100% hand recount – to provide confidence in the outcome. This paper presents the SAFE (Statistically Accurate, Fair and Efficient) alternative to percentage-based sampling, based on the same statistical principles that inform audits in business and finance. In recent federal elections, highly reliable SAFE audits would have required about the same total effort and resources as the percentage-based audits now being considered. However, SAFE audits can ensure equally high confidence in all electoral outcomes by using auditing resources more efficiently and employing large samples only when necessary. Introduction To verify election winners, Congress and several states are considering laws to require comparing machine tabulations with hand counts of paper ballots for randomly chosen precincts. 1 Since hand counts cost time and money, just enough precincts should be recounted to rule out election-altering miscounts, which may arise for a variety of reasons, including hardware malfunctions, unintentional programming errors, malicious attempts to alter election outcomes, or “undervotes” caused by ballot marks that interfere with correct counting. The key question is: how many is enough for an adequate random sample? Electoral audits, like financial or manufacturing audits, are undertaken to avoid bad outcomes – such as monetary fraud, faulty drug composition, or declaring someone to be the winner who did not get the most votes. As in finance or manufacturing, audits are equally able to detect both accidental and malicious errors. But financial audits and quality control tests set sample sizes with a quality goal in mind: specifically to be very likely to detect errors that are large enough to be harmful. In contrast, most * Verified Voting Foundation; **E-Voter Education Project; ***Political Studies Program, Bard College; ****Chair of the Subcommittee on Electoral Integrity of the American Statistical Association’s Scientific and Public Affairs Committee; *****Mathematics and Computer Science, Macalester College; ******Chair of the American Statistical Association Working Group on Accurate and Fair Elections. Please send all comments and suggestions for revision to john@verifiedvoting.org 1 One bill, H.R.811, has been reported from committee in the House of Representatives: http://thomas.loc.gov/cgi-bin/bdquery/z?d110:h.r.00811: The Chair of the Senate Rules Committee has also introduced a bill, S.1487, for consideration: http://thomas.loc.gov/cgi-bin/bdquery/z?d110:s.1487: For a list of states that already some kind of election auditing, see http://www.verifiedvoting.org/downloads/Manual_Audit_Provisions.pdf . New Jersey's legislature is currently considering a bill modeled on the SAFE approach described at the end of this paper [S.507 as amended].