Heuristic Cryptanalysis of Classical and Modern Ciphers Ho Yean Li, Azman Samsudin, and Bahari Belaton School of Computer Science, Universiti Sains Malaysia Penang, Malaysia {yeanli,azman,bahari}@cs.usm.my Abstract—Block cipher algorithms are commonly used to secure confidential information in everyday user applications However, it is quite common for ignorant users to use familiar dictionary words as their personal passwords. This research will examine the effects of weakly chosen password-keys on the security of block ciphers. A new hybrid optimization heuristic cryptanalytic attack (Tabu Search and Genetic Algorithm) is used to conduct an intelligent key-search attack on classical ciphers and modern ciphers. The algorithm chosen to represent modern block ciphers is the Advanced Encryption Standard (AES) algorithm. AES is an algebraic product cipher which combines elements of substitution and transposition. Therefore, the primarily aims of this paper is to study the effects of an optimization heuristic cryptanalytic attack on block cipher. Keywords—Cryptographic Ciphers, Cryptanalysis, and Heuristic Search. I. INTRODUCTION N today’s K-Economy where knowledge means power, cryptology is an integral part of the study of securing information and preventing confidential data from falling into the wrong hands. There are two main types of cryptographic algorithms: symmetric-key and asymmetric algorithms. Symmetric-key algorithms can be divided into two categories: block ciphers and stream ciphers. Figure 1 illustrates the different classifications of Cryptographic ciphers. This study is aimed at examining the application of a hybrid optimization heuristic cryptanalytic attack on weakly chosen keys in block ciphers; namely classical ciphers and modern ciphers. A block cipher is a symmetric-key cryptographic cipher which uses the same key to encrypt as well as decrypt fixed blocks of a secret message. There are two categories of classical ciphers: substitution ciphers and transposition ciphers. Most modern ciphers are product ciphers, which are extensions of classical ciphers. A product cipher is a block cipher which is an amalgam of components made of substitution and transposition ciphers [1]. The Advanced Encryption Standard (AES) [3] is chosen to represent the group of modern ciphers because it is a relatively new product cipher. Among all the transposition ciphers, the Columnar Transposition Cipher which is most similar to the ShiftRows step of the AES algorithm will be studied as well. Since most modern ciphers combine polygraphic substitution ciphers with a transposition cipher, the Hill Cipher is chosen to represent the substitution ciphers in this study. Furthermore, the Hill Cipher is also quite similar to the SubBytes step of the AES algorithm. Fig.1. Schematic representation of cryptographic cipher classification (adapted from [1, 2]). II. RELATED WORK A. The Substitution Cipher (The Polygraphic Hill Cipher) A substitution cipher maintains the original position of a plaintext character in the ciphertext but substitutes the value of a plaintext with another value [1]. A polygraphic Cipher substitutes blocks of characters in groups; usually pairs of characters known as bigrams. The Hill cipher is a polygraphic substitution which combines and substitutes groups of letters in a block matrix using linear algebra. According to Stallings [4], the frequency distribution of bigrams is more evenly spread in ciphers like the Hill Cipher as compared to the frequency distribution of individual letters in a monoalphabetic cipher. This makes polygraphic more difficult to break the ciphertext. It is difficult to break the Hill Cipher based on known-ciphertext only. However the linearity of the Hill Cipher makes it vulnerable to known-plaintext attacks. Hence, it is usually combined with a permutation (transposition) component as found in Modern Ciphers like Feistel Ciphers. B. The Transposition Cipher (The Columnar Transposition Cipher) The transposition cipher rearranges the positions of the plaintext characters in a different and complex order but “leaves the value of a character or character string unaltered I This work was supported by Universiti Sains Malaysia. Block Ciphers Stream Ciphers Modern Ciphers Substitution Ciphers Transposition (Permutation) Ciphers AES (Rijndael) Polyalphabetic Substitution Classical Ciphers Product Ciphers Polygraphic Substitution Mechanical Substitution Homophonic Substitution Onetime pads Columnar Transposition Double Transposition Disrupted Transposition Rail Fence Route Cipher Fractionation Hill Cipher Monoalphabetic (simple) Substitution Cryptographic Ciphers 1-4244-0000-7/05/$20.00 ©2005 IEEE.