Vol.:(0123456789) 1 3 Virtual Reality https://doi.org/10.1007/s10055-020-00426-w S.I. : XR (VR, AR, MR) AND IMMERSIVE LEARNING ENVIRONMENTS Virtual reality: an aid as cognitive learning environment—a case study of Hindi language Priyanka Jain 1  · Ram Bhavsar 2  · Karimullah Shaik 1  · Ajai Kumar 1  · B. V. Pawar 2  · Hemant Darbari 1  · Virendrakumar C. Bhavsar 3 Received: 5 February 2019 / Accepted: 4 February 2020 © Springer-Verlag London Ltd., part of Springer Nature 2020 Abstract The objective of this research is to propose a dynamic composition of behaviour-rich interactive 3D scene as a virtual environ- ment for cognitive support using visual and linguistic analytics. Based on the constructivist theory of learning through visual cognition by Winn (A conceptual basis for educational applications of virtual reality, Technical report TR 93-9, Washington Technology University, 1993), we propose our work with a grounding that visual data are easy to comprehend for a person having linguistic learning difculties. Virtual reality provides an environment for learners to actively pursue their knowl- edge needs by applying their theories in the ‘real world’. Therefore, we focus our work on generating an interactive virtual environment. It decreases cognitive load for the person with difculties in comprehension, especially in language reading, e.g. dyslexia. Our prior work related to the proposed research is named as Preksha—an automatic Hindi text visualizer. To the best of the knowledge of the authors, Preksha is the only known visualization work for an Indian Language, viz. Hindi. Belonging to morphologically-rich and free-word order Indian languages, this work on the Hindi Language is a novel inter- disciplinary approach to develop a virtual environment for cognitive support. Application of an automatic text visualization with a suitable learning paradigm in a virtual environment is another novelty of this research. Keywords Virtual reality · Visual cognition · Cognitive learning · WebVR · Artifcial intelligence · Natural language processing · Automatic text visualization 1 Introduction The objective of the proposed approach is to linguistically understand a phenomenon in the input text and to dynami- cally create a 3D virtual environment. The education of learning-disabled children demands both quantity and quality in the use of advanced technologies. Our literature survey (see Sect. 2) suggests that the learning capabilities of the user can be enhanced with visual processes, such as visualization, visual sequencing, visual memory and visual perception. The learning of a natural language is easier with visual cognition. As the abstract concepts of linguistic struc- tures are difcult to experience, it can be simplifed by the methods of visual education and real-life-like experience. We propose automatic text visualization (ATV) to decrease the cognitive load through 3D animation interface using a virtual environment. It is targeted to provide cognitive sup- port for a person with autism/mental retardation or who has difculties in comprehension, especially in language read- ing, e.g. dyslexia. Multilingual communication is required as a basic need in cross-lingual environments at national and international levels. Though the design of the proposed work is language independent, we select the Hindi language as a case study to present in this paper. In India, there are 22 ofcial lan- guages (with 1028 dialects) along with only 11 scripts to represent these languages. Therefore, language-less com- munication (to the extent possible) is ideally desired. Hindi is a national language in India, and therefore it has spread as a contemporary lingua franca and now predominates in a number of states. Since pictures, signs, gestures and * Priyanka Jain priyankaj@cdac.in 1 Centre for Development of Advanced Computing, Pune, Maharashtra, India 2 KBC North Maharashtra University, Jalgaon, Maharashtra, India 3 Faculty of Computer Science, University of New Brunswick, Fredericton, Canada