XX‐XX‐XX What's my Personal Spam Threshold (PeST)? Towards Spam Filtering as Interactive Experience Christopher Lueg (University of Tasmania, Australia) Samuel Martin (University of Tasmania, Australia) Andrew Orange (University of Tasmania, Australia) Vishv Malhotra (University of Tasmania, Australia) To appear in: Proc. Asian Conference in Information Systems (ACIS 2012) Abstract Utilizing interactive visualizations of data sets as a way of helping users explore and understand complex relationships in data has a long tradition in Human Computer Interaction, dating back at least to dynamic queries and starfield displays developed in the early Nineties. In this paper we discuss how interactive visualizations can be used to help non-expert users understand how different spam filter settings impact on what they will see in their email inboxes. The key idea is that users can visually explore the positive or negative effects of different filter settings on their email inboxes without having to know the details of how respective filtering modules work in detail. Lab-based evaluation of two prototypes suggests the approach led to an increase in the knowledge about spam filtering even among subjects possessing prior spam filtering knowledge. The work contributes to the small but growing body of work on user interaction with spam filters which is scarce even though spam filtering has become truly ubiquitous. Keywords : interfaces, interaction, interactivity, visualization, spam, scaffolding 1. INTRODUCTION Spam continues to threaten the very existence of email as we know it (e.g., Whitworth and Whitworth 2004) and continues to cost the community millions of dollars per annum. Personal email filters installed on user controlled machines and server-based email filtering and blocking mechanisms have become ubiquitous. Looking purely at effectiveness as measured by the number of false negatives (i.e., spam not classified as such) and false-positives (i.e., genuine email falsely classified as spam), it seems that spam filtering is a straight-forward activity that is relatively easy to automate. For most users, verifying the figures put forward by manufacturers of commercial spam filters may be difficult because of the sheer volume of messages that those companies filter and the fact that most messages classified as spam by those companies are removed before users even have a chance to assess their "spamminess". Unfortunately there is not a lot of research looking into user perspectives regarding spam filtering. Research dating back to 2003 suggests that 30% of the email users that were surveyed were concerned their email filters might filter genuine incoming email and 23% of users were concerned that email they send to others may be filtered (Fallows 2003). While the Fallows figure is almost a decade old, anecdotal evidence suggests that not receiving email or having to retrieve genuine messages from "spam folders" is a fairly common experience even in 2012. There is also anecdotal evidence that false classification of emails (false positives but also false negatives) increases user irritation and contributes to users making mistakes when dealing with messages. So far a spam fighter 'silver bullet' hasn't been found so that deploying the "right" mix of spam filter technologies appears to be crucial. Balvanz et al.'s (2004) experiments with a number of different spam filters built into desktop email clients revealed much higher false-positive rates than accomplished in lab tests and by professional mail filtering services. It is unfortunate that the Fallows study (ibid) did not report whether the concerns they documented were based on actual experiences with email services or merely a reflection of unspecific concerns. Furthermore, it is reasonable to assume that some of the users interviewed were using "free" web-based email services, such as hotmail.com or yahoo.com, where ways to interact with and learn about the filtering system are limited (see below). In this paper we discuss interactive exploration as a way to help users understand how different spam filter settings impact on what they see in their email inboxes. We are particularly interested in the behavior of everyday, non-expert users which complements the work by Cramer et al. (2009) providing insights into the behavior of more experienced spam filter users. The remainder of the paper is structured as follows. First, we revisit discussions examining interfaces provided by popular web email providers and in particular the information they provide to help users understand what their spam filters are doing. we are also interested in the means that they offer to users to adjust assessments of "spamminess". Then we present the evaluation of two custom-built email and spam filter interfaces, CabbageMail and Interactive Spam Filtering, both of which we developed in