AnimaChaotic: AI-based Automatic Conversion of Children’s Stories to Animated 3D Videos Reem Abdel-Salam, Reem Gody, Mariam Maher, Hagar Hosny and Ahmed S. Kaseb Computer Engineering Department, Faculty of Engineering, Cairo University, Giza, Egypt Keywords: Multimedia, AI, NLP, Storytelling, Scene Generation, Story Generation, Information Extraction. Abstract: Stories are an effective and entertaining way of teaching children about real-life experiences in an engaging way. Although many children’s stories are supplemented with graphical illustrations, having animated 3D video illustrations can effectively boost the learning process, especially for visual learners. However, produc- ing animated 3D videos is a hard, expensive, and time-consuming process, so there is a need to automate this process. In this paper, we introduce AnimaChaotic, a story visualization system designed to automatically convert children’s short stories to animated 3D videos by leveraging Artificial Intelligence (AI) and computer graphics. Our Natural Language Processing (NLP) pipeline extracts visualizable information from the story such as actors and actions. Then, our object positioning algorithm determines the initial positions of the ob- jects in the scene. Finally, the system animates the scene using different techniques including AI behaviors. A quantitative analysis of our system demonstrates a high precision and recall in extracting visualizable in- formation. It also shows that our system outperforms existing solutions in terms of static scene generation. A qualitative analysis of the system shows that its output is visually acceptable and outperforms similar solutions. 1 INTRODUCTION People have used stories for thousands of years. Sto- ries capture the attention of young children, teach them about the world around them, and summarize life experiences in a simple and engaging method. Most of the available children stories are text-based and sometimes supplemented with graphical illustra- tions to ease their understanding. This might not be enough for some children who would benefit more from a visual learning experience in which they can read, watch, and listen at the same time. Since reading is an important skill to learn at an early age, it is use- ful to have the stories supplemented with 3D video il- lustrations to encourage all the children to read while satisfying their different needs. However, video illustrations are not available for all stories as they are hard, expensive, and time- consuming to produce. This is because producing animated 3D videos involves collecting or creating necessary models for actors and objects, construct- ing scenes relevant to the story, and developing the interactions of actors and objects in the scene. All this requires expertise (from artists and developers) in using advanced software tools. That is why there is a need for a system to automate the process of con- verting stories to 3D video illustrations and make it accessible to everyone This paper proposes AnimaChaotic 1 , a system that automatically converts children’s short stories to animated 3D videos by leveraging AI and computer graphics. For an input story text, the system uses NLP techniques to extract visualizable information such as: i) Physical objects, their properties, and their constraints. ii) Actions performed by the story actors. iii) Dialogues between the actors. iv) Emotions of the actors. v) Weather conditions. vi) Background scene. Then, accordingly, the system produces the animated 3D video by constructing the scene, positioning the objects, scheduling the events, animating the scene, and overlaying the audio of the dialogues. Experiments show that the precision and recall of extracting visualizable information is more than 96% (even for the most challenging information such as indirect objects). Then, we show that our system out- performs a similar existing solution in an end-to-end comparison in terms of static scene generation. Fi- nally, we conduct a qualitative evaluation of the sys- tem using a user survey which shows that the output 1 AnimaChaotic Video Trailers and Showcases: http://www.youtube.com/watch?v=o8vY0U4bX1k http://www.youtube.com/watch?v=Jz1qfO7PtAA 226 Abdel-Salam, R., Gody, R., Maher, M., Hosny, H. and S. Kaseb, A. AnimaChaotic: AI-based Automatic Conversion of Children’s Stories to Animated 3D Videos. DOI: 10.5220/0010815100003116 In Proceedings of the 14th International Conference on Agents and Artificial Intelligence (ICAART 2022) - Volume 2, pages 226-235 ISBN: 978-989-758-547-0; ISSN: 2184-433X Copyright c  2022 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved