ISSN (e): 2250 – 3005 || Volume, 09 || Issue, 6 || June– 2019 || International Journal of Computational Engineering Research (IJCER) www.ijceronline.com Open Access Journal Page 5 Performance Evaluation of Rule Based Token Mapping Translation System for Indian Sibling Languages Hindi-Gujarati Dr. Kalyani Patel 1 , Dr. Jyoti Pareek 2 1 Assistant Professor, M.Sc (CA & IT), K.S. School of Business Management, Gujarat University, Ahmedabad 380009,Gujarat ,India. 2 Professor, Department of Computer Science, Gujarat University, Ahmedabad 380009, Gujarat , India. --------------------------------------------------------------------------------------------------------------------------------------- Date of Submission: 11-06-2019 Date of acceptance:28-06-2019 --------------------------------------------------------------------------------------------------------------------------------------- I. INTRODUCTION Machine translation means automatic translation of text by computer from one natural language into another natural language. The GH-MAP [Patel K. and Pareek J, 2013] is a rule based token mapping system for translation between sibling language pair Gujarati and Hindi. As Gujarati and Hindi are structurally similar languages, GH-MAP system generates target language sentence retaining the flavor of the source language. It should be noted that translation is not done in the sense of linguistics; instead word-by-word translation has been performed. Implementation of the GH-MAP system can be considered a success only if the quality of the translation produced by the system is of acceptable. The paper presents evaluation of the GH-MAP system using Hindi sentences extracted from FIRE 2010 1 , literature on Gandhiji [Gandhiji, 1999; 2007] and ELRA-W0037 2 . The quality of the system has been measured using prototype developed by us for automatic calculation of evaluation metrics such as Position-independent word Error Rate (PER) [Tillmann et al., 1997], B iL ingual E valuation U nderstudy (BLEU) [Papineni et al, 2000] and Metric for Evaluation of Translation with Explicit Ordering (METEOR) [Banerjee and Lavie, 2005]. II. MACHINE TRANSLATION EVALUATION TECHNIQUES Evaluation is needed to identify limitations, errors and deficiencies, which may be corrected or may be improved. In such situation human evaluation is the best option but it is impractical and costly. Evaluating a machine translation system using automatic metrics is much faster, easier and cheaper compared to human evaluations. The intuition behind metrics is that machine translation would be considered good if it resembles closely to human translation of the same sentence [Papineni et al. 2002].For our experiments, we have selected the most widespread automatic evaluation metrics based on:  Levenshtein-Based Measures: Position-independent word Error Rate (PER) [Tillmann et al.,1997] computes the Levenshtein distance without taking the word order into account.  N-Gram-Based Measures: Bilingual Evaluation Understudy (BLEU) [Papineni et al, 2000] score, measures translation quality based on precision; it compares n-gram matches between candidate translation and a reference translation.  The Importance of Recall: Metric for Evaluation of Translation with Explicit ORdering (METEOR) [Banerjee and Lavie, 2005] put more weight on recall than on precision in the harmonic mean to measure translation quality. 1 Forum for Information Retrieval Evaluation (FIRE,2010) 2 ELRA : Evaluation and Language Resources Distribution Agency, France, The EMILLE/CIIL corpus from www.elda.org ABSTRACT: The paper presents evaluation of rule based token mapping translation system for Indian Sibling languages Hindi-Gujarati. The system was evaluated on the test bed obtained from FIRE 2010, literature on Gandhiji and ELRA-W0037. For establishing relevance of the model ‘into-Gujarati’ BLEU, PER and METEOR score have been calculated. General Terms: Machine Translation, Evaluation Additional Keywords and Phrases: Sibling language, Gujarati, Hindi, GH-MAP