1 A Multimodal Sentiment Analysis Scheme to Detect Hidden Sentiments Vidula Dattatray Bhat Dept. of Information Technology Maharashtra Institute of Technology College of Engineering Pune India vidulabhat@gmail.com Vivek S. Deshpande Dept. of Information Techology Maharashtra Institute of Technology College of Engineering Pune India vivek.deshpande@mitcoe.edu.in Rekha Sugandhi Dept. of Computer Science Maharashtra Institute of Technology College of Engineering Pune India rekha.sugandi@mitcoe.edu.in Abstract- Hidden sentiments like sarcasm, irony, satire usually convey an opposite sentiment in plain text than the sentiment actually being expressed. Failure in the detection of hidden sentiments results into reduced performance of sentiment analysis system. A hidden sentiment can be detected using context and other modes of input like images, audio or video along with plain text. A context aware multimodal scheme to detect hidden sentiments to improve performance of sentiment analysis has been proposed. Key words- Multimodal sentiment analysis, opinion mining, sarcasm, irony, satire I. INTRODUCTION Today internet is rife with opinions expressed in forums, blogs, social networking sites, product and movie review sites. The opinions are not only in the form of text but also in the form of audios, images and videos. Emergence of internet as a new and powerful source of opinions is the motivation behind the study of opinion mining. The newly available source of information is huge and is being updated everyday. The data is in multiple forms- text, audio, video, images and is unstructured. Processing this data and extracting meaningful information from it is beyond human power. Thus the process needs to be automated. This is where the field of Sentiment Analysis (SA) and Opinion Mining (OM) comes into picture. Companies use sentiment analysis on product review and social networking sites to get feedback about their products. Sentiment Analysis has been also proved useful for detecting a child‟s interest in the lecture, to detect personality traits [1] and leadership traits [2]. Sentiment Analysis can also be used in gaming and for the treatment of psychological disorders. The field is not only limited to the data available on internet but can also be applied to any mode of input which expresses sentiments or opinions. Sentiment Analysis or Opinion Mining refers to the process of automatically detecting sentiments, emotions, opinions, and views from any form of expression – text, facial expressions, body language, posture, brain image mappings etc. Though Sentiment Analysis is mainly considered as a sub branch of Natural Language Processing, SA is also being used to detect sentiments from non-textual inputs. Multimodal Sentiment Analysis is an emerging field of research which combines two or more input modes. The general block diagram of a multimodal sentiment analysis system is as follows- Fig. 1 General Block Diagram of Sentiment Analysis System II. LITERATURE SURVEY Datasets available for training and testing sentiment analysis models and the algorithms used in sentiment analysis are mainly reviewed. Baveye et. al. have presented a video database LIRIS-ACCEDE consisting of 9800 video excerpts with affective annotations. The dataset was originally presented in [3] and is described in [4]. This dataset is publicly available and it is a useful resource for sentiment analysis from videos. Four experimental protocols and a baseline for prediction of emotions using audio and visual features is also provided along with video clips. Rosas et. al. [5] have implemented a multimodal sentiment analysis scheme and compared results from single input mode, combinations of two input modes, and the combination of all the considered Analysis and Prediction End Tags Hybridization Video Normalization Audio Normalization Image Normalization Text Normalization Video Text Image Audio AVCOE, Sangamner iPGCON-2015 SPPU, Pune 24th & 25th March 2015 Fourth Post Graduate Conference Page 1 of 3