IOSR Journal of Computer Engineering (IOSR-JCE) e-ISSN: 2278-0661,p-ISSN: 2278-8727, Volume 17, Issue 6, Ver. VI (Nov – Dec. 2015), PP 01-07 www.iosrjournals.org DOI: 10.9790/0661-17660107 www.iosrjournals.org 1 | Page Novel Malware Clustering System Based on Kernel Data Structure Bhandare Trupti Vasantrao 1 , Pramod B. Mali 2 1 PG Student, Department of Computer Engineering, Smt. Kashibai Navale College of Engineering, Vadgaon Bk, Pune, India 2 Assistant Professor, Department of Computer Engineering, Smt. Kashibai Navale College of Engineering, Vadgaon Bk, Pune, India Abstract : An operating system kernel is the prime of system software, responsible for the integrity and conventional computer system’s operations. Traditional malware detection approaches have based on the code- centric aspects of malicious programs, e.g. injection of unauthorized code or the control flow patterns of malware programs. In response to these malware detection strategies, modern malware focus on advanced techniques such as reusing existing code or complicated malware code to circumvent detection. A new perspective is introduced to detect malware which is different from code-centric approaches. The data centric malware defense architecture (DMDA) is introduced which models and detects malware behavior. This architecture is based on properties of the kernel data objects that are targeted during malware attacks. This architecture requires external monitoring. External monitor resides outside the monitored kernel and ensures temper-resistance. This architecture consists of three core system components that enable inspection of the kernel data properties and depending upon these properties from malware cluster. The system clusters malware depending upon the kernel data objects. Keywords: Kernel data structure, malware signature, malware clustering, kernel malware classification. I. Introduction An operating system kernel is the core of system software which is responsible for the integrity and operations of a conventional computer system. Developers of malicious software (malware) have been continuously exploring various attack vectors to tamper with the kernel. Traditional malware detection approaches have focused on the code centric aspects of malicious programs, such as the injection of unauthorized code or the control flow patterns of malware programs. However, in response to these malware detection strategies, modern malware is employing advanced techniques such as reusing existing code or complicated malware code to circumvent detection. Kernel-level malware is one of the most dangerous threats to the security of users on the Internet, so there is an urgent need for its detection. The most popular detection approach is misuse-based detection. However, it cannot catch up with today's advanced malware that increasingly apply polymorphism and obfuscation. In this thesis, we present our integrity-based detection for kernel-level malware, which does not rely on the specific features of malware. Modern malware use a variety of techniques to cause divergence in the attacked program’s behaviour and achieve the attacker’s goal. Traditional malicious programs such as computer viruses, worms, and exploits have been using code injection attacks which inject malicious code into a program to perform a nefarious function. Intrusion detection approaches based on such code properties effectively detect or prevent this class of malware attacks [10], [11], [12]. Data-centric approaches require neither the detection of code injection nor malicious code patterns. Therefore they are not directly subvertible using code reuse or obfuscation techniques. However, detecting malware based on data modifications has a unique challenge that makes it distinct from code based approaches. Unlike code, which is typically expected to be invariant, data status can be dynamic. Correspondingly, conventional integrity checking cannot be applied to data properties. In addition, monitoring data objects of an operating system (OS) kernel has additional challenges because an OS may be the lowest software layer in conventional computing environments, meaning that there is no monitoring layer below it. Traditional malware detection and analysis approaches have been focusing on code-centric aspects of malicious programs, such as detection of the injection of malicious code or matching malicious code sequences. However, modern malware has been employing advanced strategies, such as reusing legitimate code or complicated malware code to circumvent the detection. As a new perspective to complement code-centric approaches, we propose a data-centric OS kernel malware characterization architecture that detects and characterizes malware attacks based on the properties of data objects manipulated during the attacks [1].