2024 International Conference on Advances in Computing, Communication, Electrical, and Smart Systems (iCACCESS), 8-9 March, Dhaka, Bangladesh 979-8-3503-5028-9/24/$31.00 ©2024 IEEE Analyzing Abusive Bangla Comments on Social Media: NLP & Explainable AI Kahakashan Ashraf Computer Science & Engineering East Delta University Chittagong, Bangladesh Kahakashanashraf.aca@gmail.com Md.Hamid Hosen Computer Science & Engineering East Delta University Chittagong, Bangladesh mdhamidhosen4@gmail.com Safa Asgar Computer Science & Engineering East Delta University Chittagong, Bangladesh safaasgar58@gmail.com Mohammad Tanvirul Islam Computer Science & Engineering East Delta University Chittagong, Bangladesh mdtanvir1109040@gmail.com Sadia Nawar Computer Science & Engineering East Delta University Chittagong, Bangladesh sadianawar2019@gmail.com Abstract—Popularity of Social media sites has been on the rise ever since its invention. Due to the anonymity the social media sites provide to the users, an influx of divergent behavior has also been increasing. Abusive language and offensive content are posted on these sites to ridicule others which creates a harmful environment for the people affected. Detecting these misconducts, especially for Bengali Language is still vague. Our paper tries to bridge the gap between the abusive content detection using Natural Language Processing with five different machine learning classifiers such as Decision Tree, Random Forest, Multinomial Naïve Bayes, Support Vector Classifier and Logistic Regression and give clarity on the factors that contribute to the prediction using Explainable AI. The dataset was preprocessed in multiple steps including stop words and punctuation removal, null and duplication value check and tokenization. The dataset was then processed to created TF-IDF representations with n-grams. Grid Search Cross Validation technique was used to find out optimal combination of hyperparameters for our machine learning models. Among all the models Logistic Regression performed the best with an accuracy of 85.57%. For transparency and clarification of the model’s detection methodologies Lime model was used. Keywords—Machine Learning, NLP, XAI, Bangla Comments, Online Abuse I. INTRODUCTION In this digital age, social media platforms have become a popular medium for expressing thoughts, opinions, and emotions. Albeit a valuable tool for connecting with others, it also comes with a number of challenges, especially around abusive language and offensive content [1], [2]. Facebook a leading platform with a diverse user base, representing over a hundred languages and a wide range of cultural expressions. Online communities and recipients can be negatively affected by abusive comments [3]. As a result, people may feel isolated, harassed, or even physically harmed [4]. Additionally, abusive comments can create a hostile environment that discourages participation and restricts freedom of speech [5]. Natural language processing (NLP) and explainable artificial intelligence (XAI) can play a significant role in addressing the issue of online abuse. NLP techniques can be utilized to automatically identify abusive comments, while XAI can help us understand the factors that lead to online abuse. It is also possible to detect malicious intent in online conversations with NLP and analyze the motivations behind online abuse with XAI. As a result, more effective interventions can then be developed to prevent and mitigate online abuse. The prevalence of abusive language in online spaces creates a significant concern, impacting users' well-being and the overall health of online communities. It is challenging to analyze offensive and normal Bangla comments on Facebook. NLP models have trouble identifying abusive language in Bangla because it has complex sentence structures and a rich morphology. Secondly, abuse definition can be subjective and context-dependent, making it difficult to create a reliable dataset for train NLP models. Moreover, even well-trained NLP models are not always able to explain why a particular comment is abusive. Although NLP models are capable of predicting, they lack transparency in their decision-making processes, causing them to be untrustworthy and interpretable. In this study, we present a framework for analyzing Bangla comments on Facebook, comparing abusive and non-abusive language, and providing insights into the reasoning behind these classifications. Our paper introduces a framework for analyzing Bangla comments on social media, to distinguish between abusive and non-abusive language using Natural Language Processing (NLP) techniques, and the LIME (Local Interpretable Model- agnostic Explanations) framework. Our research seeks to enhance the transparency and interpretability of the NLP model's decision-making process. The specific goals include optimizing NLP-based classification of Bangla comments, integrating LIME to generate locally faithful explanations for individual predictions, and conducting a detailed analysis of linguistic patterns to provide nuanced insights into the identification of abusive language. This research aims to contribute to a more interpretable and accountable framework for the detection and understanding of online abusive language in the Bangla language context. The paper is divided into multiple sections. In Section II we discuss the related works, Section III demonstrates our proposed architecture and methods included. Section IV discusses the performance of the models and evaluation metrices. Lastly, Section V provides a comprehensive