International Journal of Electrical and Computer Engineering (IJECE) Vol. 13, No. 3, June 2023, pp. 2921~2930 ISSN: 2088-8708, DOI: 10.11591/ijece.v13i3.pp2921-2930  2921 Journal homepage: http://ijece.iaescore.com Flagging clickbait in Indonesian online news websites using fine- tuned transformers Muhammad Noor Fakhruzzaman 1 , Sa’idah Zahrotul Jannah 2 , Ratih Ardiati Ningrum 1 , Indah Fahmiyah 1 1 Data Science Technology Study Program, Faculty of Advanced Technology and Multidiscipline, Universitas Airlangga, Surabaya, Indonesia 2 Statistics Study Program, Faculty of Science and Technology, Universitas Airlangga, Surabaya, Indonesia Article Info ABSTRACT Article history: Received Aug 15, 2022 Revised Sep 6, 2022 Accepted Oct 1, 2022 Click counts are related to the amount of money that online advertisers paid to news sites. Such business models forced some news sites to employ a dirty trick of click-baiting, i.e., using hyperbolic and interesting words, sometimes unfinished sentences in a headline to purposefully tease the readers. Some Indonesian online news sites also joined the party of clickbait, which indirectly degrade other established news sites' credibility. A neural network with a pre-trained language model multilingual bidirectional encoder representations from transformers (BERT) that acted as an embedding layer is then combined with a 100 node-hidden layer and topped with a sigmoid classifier was trained to detect clickbait headlines. With a total of 6,632 headlines as a training dataset, the classifier performed remarkably well. Evaluated with 5-fold cross-validation, it has an accuracy score of 0.914, an F1-score of 0.914, a precision score of 0.916, and a receiver operating characteristic-area under curve (ROC-AUC) of 0.92. The usage of multilingual BERT in the Indonesian text classification task was tested and is possible to be enhanced further. Future possibilities, societal impact, and limitations of clickbait detection are discussed. Keywords: Adult literacy Clickbait Natural language processing Online news This is an open access article under the CC BY-SA license. Corresponding Author: Muhammad Noor Fakhruzzaman Data Science Technology Study Program, Faculty of Advanced Technology and Multidiscipline Universitas Airlangga St. Airlangga No. 4-6, Airlangga, Surabaya, East Java 60115, Indonesia Email: ruzza@ftmm.unair.ac.id 1. INTRODUCTION Journalism has changed. Before the emergence of internet news, we bought newspapers because we were enticed by the headline on the front page which usually leads to the truth, but not anymore. The emergence of online news outlets created a whole new scheme for making money in the journalism world, the online ad. With internet advertising, a single click means money, even though it is not as much as a newspaper sale or advertising money from sponsors, like in the olden days. Now, a post headline has to rake in engagement, the metric that measures ratings in online news. The scheme of online advertising that bases on engagement have a negative influence on the original journalism idea. Sadly, online news organization now hunts for click money instead of the truth. This phenomenon promotes a unique style of headline writing, infamously known as clickbait. The more people click the post, the more engagement that post has, and the more advertising value the site will gain. A study found that most online news organization relies on clickbait's ad money to support their daily activities. With an increasing number of online news sites in recent years, they have to contest for reader's clicks [1]–[4].