A New Hybrid Approach of Symmetric/Asymmetric Authentication Protocol for Future Mobile Networks Mustafa Al-Fayoumi , Shadi Nashwan, Sufian Yousef, *Abdel-Rahman Alzoubaidi Anglia Ruskin University, Chelmsford-UK Al-Zaytoonah University, Amman-Jordan maa135@student.anglia.ac.uk , sn224@student.anglia.ac.uk , s.yousef@anglia.ac.uk, *zoubaidi@alzaytoonah.edu.jo Abstract Most of current authentication schemes for mobile systems have some weaknesses; such as leakage of UE identities and high update overhead of temporary identities. This paper proposes a secure authentication mechanism for mobile communication systems that satisfies the security requirements of the third generation mobile systems. In this proposed protocol, the number of messages between authentication entities of the network is reduced to four messages instead of five in initial authentication procedure. The subsequent authentication procedure only contains two message exchanges. Therefore, the bottleneck at authentication center is avoided by reducing the number of messages between mobile and authentication center. The authentication time delay, call setup time and signaling traffic are minimized. Also, this proposed protocol is designed to be secure against network attacks, such as replay attacks and Guessing attacks and others attacks. Consequently, this approach is secure and practical as it can satisfy the security requirements of the third generation mobile communication systems based on hybrid asymmetric and symmetric cryptosystem, and can save up to 20% of the authentication traffic delay time. KEYWORDS: 3rd Generation Mobile Networks, Authentication mechanism, End-to-End mobile security and Mobile Security Requirements. 1. Introduction With the recent advances in wireless communication and computer technologies, the volume and speed of information collection and processing have dramatically increased. Wireless networks rapidly evolve from actual 2G cellular telephony network to 3G and beyond such that the mobile communications have become more convenient than ever. Nowadays, people can communicate with each other at any place at any time. However, because of the openness of wireless communications, protecting the privacy between communicating parties is becoming a very important issue [6, 7]. Since the transmission between users and the mobile communications system are air interface, security is one of the most important requirements of mobile communication systems and authentication protocol is the most necessary procedure to ensure that the service is properly used. The authentication protocol is a communication process that all the participants ensure their legality and verify other participant’s identities involved in the mobile communication systems. Hence, based on the authentication protocol, we can reduce or eliminate threats that eavesdropping and masquerading legal users. However, it only authenticates part of participants, not all the communication participants. Authentication protocols supply the basic methods for implementation of security services. An authentication protocol is designed to allow participating entities that demonstrate its knowledge of certain secrets, including ensure the identities of all parties over wireless link and establish a common secret key between them [8]. The first public GSM was created on the 1st July 1991, which is regarded as the second-generation mobile telecommunication [4, 5]. In the past ten years, GSM has become a truly universal mobile communication system. The 2G systems mainly provide speech services. Hence, ten years later, GSM has brought us onto the footprint of the third generation mobile communications system, which is Universal Mobile Telecommunication System (UMTS) in European. Recently, several advanced mobile communication systems have become the standards for third-generation mobile communication Third IEEE International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob 2007) 0-7695-2889-9/07 $25.00 © 2007