Towards emotion recognition in immersive virtual environments: A method for Facial emotion recognition Kahina Amara 1 , Naeem Ramzan 2 , Nadia Zenati 1 , Oualid Djekoune 1 , Cherif Larbes 3 , Mohamed Amine Guerroudji 1 and Djamel Aouam 1 1 Centre of developpement of advanced technologies Algiers, Algeria 2 School of Engineering and Computing, University of the West of Scotland Paisley, Scotland 3 ENP Ecole Nationale politechnique Hassen Badi Avenue, Algiers, Algeria Abstract Virtual Reality (VR) is, thus, proposed as a powerful tool to simulate complex, real situations and envi- ronments, ofering researchers unprecedented opportunities to investigate human behaviour in closely controlled designs in controlled laboratory conditions. Facial emotion recognition has attracted a great deal of interest for interaction in virtual reality, healthcare system: therapeutic applications, surveillance video application etc. In this paper, we propose a method for facial emotion recognition for immersive virtual environment based on 2D and 3D geometrical features. We used our collected dataset of 17 sub- jects’ performance of six basic facial emotions (anger, fear, happiness, surprise, sadness, and neutral) using three devices: Kinect (v1), Kinect (v2), and RGB HD camera. In addition, we present the perfor- mance results of the RGB data for facial emotion recognition using Bagged Trees algorithm. To assess the performance of the proposed system, we used leave-one-out-subject cross-validation. We compared the 2D and 3D data performance for facial expression recognition. The obtained results show the su- perior performance of the RGB-D features provided by Kinect (v2). Our fndings highlight that the 2D images are not robust enough for facial emotion recognition. The built facial emotion models will an- imate virtual characters that can express emotions via facial expressions. This could be deployed for Chatting, Learning and Therapeutic Intervention. Keywords Virtual Reality, Facial emotion recognition, Immersive Environment, Avatar animation, Interaction, RGB, RGB-D, Machine Learning, Geometrical features ICCSA’21: The 2nd International Conference on Complex Systems and their Applications, May 25-26, 2021, Oum El Bouaghi, Algeria  kamara@cdta.dz (K. Amara); Naeem.Ramzan@uws.ac.uk (N. Ramzan); nzenati@cdta.dz (N. Zenati); odjekoune@cdta.dz (O. Djekoune); cherif.larbes@g.enp.edu.dz (C. Larbes); mguerroudji@cdta.dz (M. A. Guerroudji); aouam@cdta.dz (D. Aouam)  0000-0001-6673-0143 (K. Amara) © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org)