Multimedia Tools and Applications https://doi.org/10.1007/s11042-018-6755-1 An intelligent recommendation system using gaze and emotion detection Saurabh Jaiswal 1 · Shubham Virmani 1 · Vishal Sethi 1 · Kanjar De 1 · Partha Pratim Roy 1 Received: 8 December 2017 / Revised: 9 August 2018 / Accepted: 4 October 2018 / © Springer Science+Business Media, LLC, part of Springer Nature 2018 Abstract Recently, recommendation system has become popular in many e-commerce websites. It helps users by suggesting products which they could buy. Existing work till now uses past feedback of user, similarity of other users’ buying pattern, or a hybrid approach in which both type of information is used. But the pitfall of these approaches is that there is a need to collect and process huge amount of data for good recommendation. This paper is aimed at developing an efficient recommendation system by incorporating user’s emotion and interest to provide good recommendations. The proposed system does not require any of aforementioned data and works without the continuous and interminable attention of the user. In this framework, we capture user’s eye-gaze and facial expression while exploring websites through inexpensive, visible light “webcam”. The eye-gaze detection method uses pupil-center extraction of both eyes and calculates the reference point through a joint prob- ability. The facial expression uses landmark points of face and analyzes the emotion of the user. Both methods work in approximate real time and the proposed framework thus pro- vides intelligent recommendations on-the-fly without requirement of feedback and buying patterns of users. Keywords E-commerce recommendation system · Eye gaze · Emotion detection 1 Introduction Human-computer interaction has become an increasingly important part of our daily lives. In today’s world, computer affects and facilitates nearly every aspect of modern life [16]. From being used as a tool to calculate complex mathematical equations, this modern mar- vel has transformed into a device to learn new things, shop online and even connect with people who are millions of kilometers apart. With all such facilities and services, this digital equipment has become a host to a vast proportion of the human population.  Partha Pratim Roy proy.fcs@iitr.ac.in 1 Department of Computer Science and Engineering, Indian Institute of Technology Roorkee, Roorkee 247667, India