A preliminary two-stage alarm correlation and filtering system using SOM neural network and K-means algorithm Gina C. Tjhai*, Steven M. Furnell, Maria Papadaki, Nathan L. Clarke Centre for Security, Communications and Network Research, University of Plymouth, Plymouth PL4 8AA, United Kingdom article info Article history: Received 21 July 2009 Received in revised form 5 February 2010 Accepted 25 February 2010 Keywords: Intrusion Detection System False alarm Self Organising Map (SOM) K-means clustering Alarm correlation abstract Intrusion Detection Systems (IDSs) play a vital role in the overall security infrastructure. Although the IDS has become an essential part of corporate network infrastructure, the art of detecting intrusion is still far from perfect. A significant problem is that of false alarms, as generating a huge volume of such alarms could render the system inefficient. In this paper, we propose a new method to reduce the number of false alarms. We develop a two- stage classification system using a SOM neural network and K-means algorithm to corre- late the related alerts and to further classify the alerts into classes of true and false alarms. Preliminary experiments show that our approach effectively reduces all superfluous and noisy alerts, which often contribute to more than 50% of false alarms generated by a common IDS. ª 2010 Elsevier Ltd. All rights reserved. 1. Introduction Networked systems have become increasingly prevalent, fast, and inexpensive, leading to a rapid growth in both demand and complexity of the computing system. Unfortunately, this has also been accompanied by a growth in the threats to the systems. In 2008, the number of new malicious code signa- tures increased over 265 percent over 2007; more than 60 percent of the total code threats were detected in 2008, as reported by Symantec (2009). The huge increase in the number of malicious code threats demonstrates the growing need for more responsive and reliable security measures. An Intrusion Detection System (IDS) is a component of a network security architecture, which involves the monitoring of computer systems for intrusive activities (i.e. those behav- iours that infringe the established security model). The rise of cybercrime on the global network has entailed a great demand for a remarkable use of IDS, which in turn forms the necessity of developing a better detection system. Although IDS has become an essential part of a corporate network infrastructure, the art of detecting intrusion is still far from perfect. IDS tends to generate a huge amount of alerts, which can be mixed with false alarms. False alarms, also known as false positives (Type I errors), occur when a legitimate activity has been mistakenly classified as malicious by the IDS. The vast imbalance between the actual and false alarms generated has undoubtedly undermined the performance of IDS (Chyssler et al., 2004b). An alarm reduction system is an absolute need for this problem. This paper proposes a two-stages clustering system to reduce false alarm rate. The proposed method can classify the alarms generated by the IDS into false and true alarms. The main objective of this approach is to correlate the alerts into a more manageable form before they are presented to the adminis- trator; and to reduce the sheer volume of alerts generated. Section 2 provides a critical analysis of existing research in the area. The framework of the proposed system is presented in Section 3, whilst the concept and algorithm of the meth- odology is described in Section 4. Section 5 discusses the * Corresponding author. E-mail address: info@cscan.org (G.C. Tjhai). available at www.sciencedirect.com journal homepage: www.elsevier.com/locate/cose computers & security 29 (2010) 712 e723 0167-4048/$ e see front matter ª 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.cose.2010.02.001