148 International Journal of Communication Networks and Information Security (IJCNIS) Vol. 11, No. 1, April 2019 Application-Based Authentication on an Inter-VM Traffic in a Cloud Environment Karim Benzidane, Sâad Khoudali, Leila Fetjah, Said Jai Andaloussi and Abderrahim Sekkaki Computer Science Department, Laboratory of Research in Computer Science and Innovation University Hassan II, Faculty of Sciences Ain Chock, Casablanca, Morocco Abstract: Cloud Computing (CC) is an innovative computing model in which resources are provided as a service over the Internet, on an as-needed basis. It is a large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet. Since cloud is often enabled by virtualization and share a common attribute, that is, the allocation of resources, applications, and even OSs, adequate safeguards and security measures are essential. In fact, Virtualization creates new targets for intrusion due to the complexity of access and difficulty in monitoring all interconnection points between systems, applications, and data sets. This raises many questions about the appropriate infrastructure, processes, and strategy for enacting detection and response to intrusion in a Cloud environment. Hence, without strict controls put in place within the Cloud, guests could violate and bypass security policies, intercept unauthorized client data, and initiate or become the target of security attacks. This article shines the light on the issues of security within Cloud Computing, especially inter-VM traffic visibility. In addition, the paper lays the proposition of an Application Based Security (ABS) approach in order to enforce an application-based authentication between VMs, through various security mechanisms, filtering, structures, and policies. Keywords: Cloud Computing, Security, Inter-VM, DPI, DPDK, Blockchain, Inter-VM traffic, Virtualization. 1. Introduction According to ISO subcommittee 38, the CC study group, Cloud Computing (CC) is a paradigm for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable cloud resources accessed through services which can be rapidly provisioned and released with minimal management effort or service provider interaction [1]. It has successfully managed to advertise itself as one of the fastest growing service models. For organizations, the clouds per-use approach provides tangible relief from hardware or software investments by offering a pay-for-service model. As an extension of Grid Computing and Distributed Computing, CC aims to provide users with flexible services in a transparent manner. The benefits of CC include greater resource access, dynamic scaling, and improved costs, along with the ease of automated management for resources and performance. Consumers adopt cloud computing to reduce infrastructure overhead, adjust service levels to meet changing needs, and to quickly deliver applications. CC relies on multi-tenant environments where multiple clients are served by one software instance. It offers scaled performance and services based on shared resources, including databases, other applications, and OSs. For some organizations, this leaves them open to a variety of threats both from inside the firewall, as in the case of a private cloud, and from outside. The major roadblock to full adoption of CC has been concern regarding the security and privacy of information. Furthermore, attackers can exploit the large amount of resources in a cloud for their advantage. Network security is an important subject that is defined by protection of valuable resources such as services and information in the network. An intrusion is a group of actions that try to affect this security and consequently damage confidentiality, integrity or availability of resources. Therefore, providing security in a distributed system requires more than user authentication with passwords or digital data transmission. In fact, due to it distributed nature, CC makes it an easy target, vulnerable and prone to sophisticated attacks. Often the most utilized technology to implement a Cloud environment is virtualization with a massive multi-tenancy usage; it opens a door to a whole other level of security issues. The security factor of a CC infrastructure is an important one. In this case, we need to look into the Cloud environment (Network, Systems and applications) in a very deep manner to keep it informant, that will help to prevent security and performance disruptions which can destroy and compromise smooth transactions, and since the main concept of the CC is to allow end users to execute various applications in on-premise or off-premise resources, most of them are shared in a virtual environment creating new security and performance challenges leveraging attacks to be launched from compromised Virtual Machines (VM) that can damage the ability to serve all end users demands. In a virtual environment, there are several VMs hosted on a single physical server or hypervisor, where communication level issues can be identified either at the network level, host level and application level [2]. VMs generally inter-communicate with each other’s via virtual switch without leaving the server, and this introduces the network blind spot letting any network security appliance set on the LAN blind to any communication between VMs and this could be within a single host and also across physical servers. If the traffic doesn’t need to pass through that security appliance mostly a firewall, opening a loophole for all sorts of security attacks. Thus, the starting point of an attacker is compromising only one VM and using it as a springboard to take control of the other VMs within the same hypervisor (a technique called VM hopping or jumping) and this is generally done without being monitored or detected, giving the attacker a huge hack