IJCSNS International Journal of Computer Science and Network Security, VOL.16 No.8, August 2016 55 Manuscript received August 5, 2016 Manuscript revised August 20, 2016 RSA Encryption Algorithm Optimization to Improve Performance and Security Level of Network Messages Fausto Meneses, Walter Fuertes, José Sancho, Santiago Salvador, Daniela Flores, Hernán Aules, Fidel Castro Jenny Torres Alba Miranda, Danilo Nuela Universidad de las Fuerzas Armadas Sangolquí, Ecuador Escuela Politécnica Nacional Quito, Ecuador Universidad Técnica de Ambato Ambato, Ecuador Summary Asymmetric cryptographic algorithms are a robust technology used to reduce security threats in the transmission of messages on the network. Nowadays, one of the disadvantages are the mathematical solutions because they require a greater amount of calculation that leads to the need for increased use of computational resources. This paper aims to optimize the RSA encryption algorithm and thus improve the security, integrity and availability of information. The results show the efficiency and functionality of the RSA algorithm in terms of information security. Also, we can see that time, memory, processor and network performance when performing encryption and decryption are lower than other RSA solutions, because calculations are performed on the client and server. Key words: RSA; RPC; performance; security; asymmetric encryption. 1. Introduction Encryption and decryption of information has proven to be the best way to get confidentiality and integrity of data. Nevertheless, there is a big challenge since threats and vulnerabilities are increasing with the development of technologies [1]. In order to face this problem, the scientific community has emphasized their skills in finding an alternative to improve information security by ensuring the information availability. Nowadays, different algorithms have been promoted to provide security but at the same time, generates a higher cost and consumption of computational resources. One of these mechanisms is the RSA asymmetric encryption. RSA is the most widely used worldwide algorithm, which provides security through encryption of data that transit in the Web and ensures information confidentiality and authenticity [2]. This algorithm also known as public key algorithm, became very popular due to its simplicity in calculation. However, the security of the RSA algorithm depends on the size of the prime numbers used in factorization. It is affected by the increase of computational cost [3][4] related with prime factorization, which implies bigger key length to ensure security. The development of this work includes the study of modeling techniques in order to determine which one is feasible to represent the information model. These techniques include standard modeling languages such as Unified Modeling Language (UML); frameworks such as the Model-Driven Architecture (MDA); and network management, for example, the RSA model. Subsequently, the existing approaches for modeling encryption systems are studied. With these results, we designed a model for encryption and decryption of information based on RSA. Finally, for validation, in the Application Programming Interface (API) implementation we made a library that allows us to encrypt the message in the client side and send it together with public keys through the network. The data is retrieved on the server side and through the access to the database, private keys are recovered (decryption process). To optimize the security of the model, private keys are periodically updated through a mixing process. In order to measure the level of efficiency of the proposed model, another model was designed and implemented, called in this study Baseline RSA Model, which works with the factorization of 300 digit prime numbers. The main contribution of this research is the development of a mathematical and software optimized model that provides the following improvements: (1) the application of a mathematical model that combines modular and probabilistic calculation; (2) a matrix capable of generating encrypted messages with the same information value, but with different meanings; (3) a mixing process for updating private key; (4) the management of messages through a RPC; (5) the conversion of a deterministic basis project to a probabilistic project with the generation of random values; (6) the work with less complex structures reducing the consumption of time and resources; and (7) increases security by hiding private keys in the executable file. The remainder of this paper has been structured in the following way. Section II describes the works found in literature related with the research. Section III describes the definition and the statement of the problem. Section IV compares the baseline RSA model with the Optimized RSA model. Section V presents the analysis of the results