International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395 -0056 Volume: 03 Issue: 03 | Mar-2016 www.irjet.net p-ISSN: 2395-0072 © 2016, IRJET ISO 9001:2008 Certified Journal Page 1926 Bidirectional Hadoop Kakfa Managing Messaging Bus Prachi M. Birajdar 1 , Kanchan Ujede 2 , Rohini Yalawar 3 , Kailas H. Biradar 4 , Zeeshan Khan 5 , Swapnil Chaudhari 6 1234 Student, Computer Engineering Department, MMIT, Maharashtra, INDIA 5 Cloud Automation Java Developer , Maharashtra, INDIA 6 Professor, Computer Engineering Department, MMIT, Maharashtra, INDIA ---------------------------------------------------------------------***--------------------------------------------------------------------- Abstract - In this paper, We introduce Kafka, a robust messaging system that we developed for collecting and delivering high volumes of data. Our system incorporates ideas from existing messaging systems, and is suitable for message consumption. Using hadoop framework which stores large amount of unstructured data. To develop bidirectional communication via kafka connector ,the data can be send without any data loss. Our expected results will produce superior performance of kafka connector when compared with two popular messaging system. Key Words: Key word1, Hadoop , Kafka , zookeeper 1. INTRODUCTION Hadoop is a combination of two domain that is distributed system and bigdata. Hadoop has 100% guarantee to secure of data and availability. In hadoop echo system data at one place and code will move to the data. 1.1. HDFS The Hadoop Distributed File System is based on the Google File System and provides a distributed file system that is designed to run on commodity hardware. It has many similarities with existing distributed file systems. However, the differences from other distributed file systems are significant. It has to be low cost hardware and fault tolerant. It provides high throughput access to application data and is suitable for applications having large datasets. Fig-1 : HDFS architecture 1.2 Zookeeper Zookeeper is a thread and also demon process. DataNode:actual work done by it. NameNode If namenode will dead then it will give task on one of the datanode which is nearer to that namenode. 1.3 Hbase It is combination of hadoop distributed file system. It is used to store all the information about entire the echo system.