Research Article Two-Way Feature Extraction Using Sequential and Multimodal Approach for Hateful Meme Classification Apeksha Aggarwal, 1 Vibhav Sharma, 1 Anshul Trivedi, 1 Mayank Yadav, 1 Chirag Agrawal, 1 Dilbag Singh, 1 Vipul Mishra, 1 andHass` ene Gritli 2,3 1 Department of Computer Science and Engineering, Bennett University, Greater Noida 201310, India 2 Higher Institute of Information and Communication Technologies, University of Carthage, Tunis, Tunisia 3 RISC Lab (LR16ES07), National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia Correspondence should be addressed to Hass` ene Gritli; grhass@yahoo.fr Received 13 February 2021; Revised 6 April 2021; Accepted 8 April 2021; Published 19 April 2021 Academic Editor: M. Irfan Uddin Copyright © 2021 Apeksha Aggarwal et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Millions of memes are created and shared every day on social media platforms. Memes are a great tool to spread humour. However, some people use it to target an individual or a group generating offensive content in a polite and sarcastic way. Lack of moderation of such memes spreads hatred and can lead to depression like psychological conditions. Many successful studies related to analysis of language such as sentiment analysis and analysis of images such as image classification have been performed. However, most of these studies rely only upon either one of these components. As classifying meme is one problem which cannot be solved by relying upon only any one of these aspects, the present work identifies, addresses, and ensembles both the aspects for analyzing such data. In this research, we propose a solution to the problems in which the classification depends on more than one model. is paper proposes two different approaches to solve the problem of identifying hate memes. e first approach uses sentiment analysis based on image captioning and text written on the meme. e second approach is to combine features from different modalities. ese approaches utilize a combination of glove, encoder-decoder, and OCR with Adamax optimizer deep learning algorithms. Facebook Challenge Hateful Meme Dataset is utilized which contains approximately 8500 meme images. Both the approaches are implemented on the live challenge competition by Facebook and predicted quite acceptable results. Both approaches are tested on the validation dataset, and results are found to be promising for both models. 1.Introduction In the present era, social media is the most important activity that directly or indirectly affects people [1]. Although social media is a great platform to masses for developing skills, reach to experts, and for expressing talent, this platform has helped many people to gain success by sharing and esca- lating their work around the globe with the Internet. Sharing of memes on social media is increasing rapidly. Memes spread humour on a positive side. However, technology comes with a boon and a bane. ese memes on the negative side can hurt any group or an individual. Internet memes can most commonly be defined as still images with text that spread rapidly among people and become a craze. ey attempt to make us laugh at the expense of a theme or a person. ey often carry a deeper meaning. Memes can be made by anyone. A section of audience may find them funny while another section may find them offensive. Memes are widely spread in social media sites such as Quora, Instagram, Twitter, Facebook, Snapchat, and WhatsApp. Memes are a great tool to spread humour; however, some people use it to target an individual or a group and to offend them in a polite and sarcastic way. Such memes spread hatred, and their excess may lead to depression. Nowadays, memes are made on countless topics like politics, movies, games, college life, and comic book characters. In this work, we are addressing a real-world problem by using multiple techniques of deep learning. Most research is Hindawi Complexity Volume 2021, Article ID 5510253, 7 pages https://doi.org/10.1155/2021/5510253