Bulletin of Electrical Engineering and Informatics Vol. 13, No. 1, February 2024, pp. 482~489 ISSN: 2302-9285, DOI: 10.11591/eei.v13i1.5263  482 Journal homepage: http://beei.org IndoPolicyStats: sentiment analyzer for public policy issues Muhammad Noor Fakhruzzaman 1 , Sa’idah Zahrotul Jannah 2 , Sie Wildan Gunawan 1 , Angga Iryanto Pratama 1 , Denise Arne Ardanty 1 1 Department of Advanced Technology, Faculty of Advanced Technology and Multidiscipline, Universitas Airlangga, Surabaya, Indonesia 2 Department of Mathematics, Faculty of Science and Technology, Universitas Airlangga, Surabaya, Indonesia Article Info ABSTRACT Article history: Received Nov 11, 2022 Revised Jul 14, 2023 Accepted Aug 2, 2023 The government requires some vaccination for public health. This has led to a debate in recent years, especially during the Covid-19 pandemic. This research aims to analyze the two sentiments of the public regarding the vaccination policy. This would be helpful to ensure the acceptance of the government campaign about vaccination. The data used was text data obtained from Twitter when Indonesia was facing the second wave of the Covid-19 pandemic. The data were pre-processed by removing noise data, case folding, stemming, and tokenizing. Then, the data were classified with random forest, Naïve Bayes, and XGBoost. The results showed that all classifiers exhibit satisfying performance but XGBoost performs slightly better in accuracy value. This method can be deployed to be an automatic sentiment analyzer to help the government understand public feedback about its policies. This would be given by proper pre-processing and enough datasets. Keywords: Covid-19 Pandemic Public policy Sentiment analysis Vaccination This is an open access article under the CC BY-SA license. Corresponding Author: Sa’idah Zahrotul Jannah Department of Mathematics, Faculty of Science and Technology, Universitas Airlangga Kampus Merr C, Jl. Dr. Ir. H. Soekarno, Mulyorejo, Surabaya, Indonesia Email: s.zahrotul.jannah@fst.unair.ac.id 1. INTRODUCTION Vaccination requirements issued by the government have always created debate among the people throughout decades [1]–[3]. The various responses ranged from positive sentiments about the government’s good intent, to negative conspiracy theories [4]. It is important to understand what are the aspects that influence people’s sentiments, as they can be used to strengthen the power of the next public health campaign [5], [6]. To ensure the acceptance of the government’s campaign, especially public health-related campaigns, we must first understand the socio-cultural background of the people, or the audience [7]–[9]. During the Covid-19 pandemic in Indonesia, there are notably two sides of the spectrum: the conspiracy theorists and the positivists. Hence, it is important to analyze the two sentiments of the public regarding the vaccination policy. Considering the problems with Indonesians, social media, and how it has such a strong influence on the public’s general opinion, this research aims to establish a formal analysis of the public discourse about vaccination policy. Focusing on Indonesia, this study is trying to identify the various sentiment about vaccination policy, specifically to understand what aspects and topics underlie those sentiments. As mentioned in [5]–[7] understanding the theme that drives the various sentiments could give information for better future vaccination campaigns, as similar topics can be used for new campaign material. This research utilized well-tested baseline models for classifying the sentiments, such as Naive Bayes, XGBoost, and random forest classifiers. Also using term frequency-inverse document frequency (TF-IDF) to measure each word's importance in each tweet and the whole dataset. TF-IDF has been used by