Social Media Manipulation Awareness through Deep Learning based Disinformation Generation Clara Maathuis 1 and Iddo Kerkhof 2 1 Open University of the Netherlands, Heerlen, The Netherlands. 2 NLP Software Engineer, The Netherlands. clara.maathuis@ou.nl email@iddo.eu Abstract: As a digital environment introduced for establishing and enhancing human communication through different social networks and channels, social media continued to develop and spread at an incredible rate making it difficult to find or imagine a concept, technology, or business that does not have or plan to have its social media representation and space. Concurrently, social media became a playground and even a battlefield where different ideas carrying out diverse validity degrees are spread for reaching their target audiences generated by clear and trustable well-known, uncertain, or even evil aimed entities. In the stride carried out for preventing, containing, and limiting the effects of social manipulation of the last two types of entities, proper/effective security awareness is critical and mandatory in the first place. On this behalf, several strategies, policies, methods, and technologies were proposed by research and practitioner communities, but such initiatives take mostly a defender perspective, and this is not enough in cyberspace where the offender is in advantage in attack. Therefore, this research aims to produce social media manipulation security awareness taking the offender stance by generating and analysing disinformation tweets using deep learning. To reach this goal, a Design Science Research methodology is followed in a Data Science approach, and the results obtained are analysed and positioned in the ongoing discourses showing the effectiveness of such approach and its role in building future social media manipulation detection solutions. This research also intends to contribute to the design of further transparent and responsible modelling and gaming solutions for building/enhancing social manipulation awareness and the definition of realistic cyber/information operations scenarios dedicated/engaging large multi-domain (non)expert audiences. Keywords: information operations, cyber operations, social manipulation, disinformation, misinformation, security awareness, machine learning, deep learning. 1. Introduction “The success of manipulation depends on the level of conviction and force of the denial.” (Tess Binder) It would be difficult to recall a modern or current societal and technological trend, topic, or event that is not projected in the digital realm through social media discourses and not surrounded by diverse manipulation mechanisms like disinformation and misinformation (Maathuis & Chockalingam, 2022b). In this realm, agents, e.g., public opinion organizations, independent bodies, and civilians carry out different activities and engage (un)intentionally with diverse manipulation forms which impact their behaviour individually and collectively (Bastick,2021; Chockalingam & Maathuis, 2022). Engaging the target audience through social manipulation can be done through vectors, e.g., user profiling being (i) demographic meaning identifying ind ividuals’ unique characteristics, beliefs, needs, and vulnerabilities, and (ii) psychometric implying personality-based segmentation, and micro-targeting based on analyzing/altering audience’s personal actions (Kertysova, 2018; Fard & Maathuis, 2021). While efforts dedicated to limiting, controlling, and preventing social manipulation using AI-based techniques, e.g., Twitter and Facebook relying on Machine Learning-based solutions for stamping out trolls, finding and removing fake bot accounts, and proactively identifying sensitive content (Perez-Escolar et al. 2021), still these mechanisms are insufficient as the attackers are intelligent in developing adaptive techniques that succeed on bypassing defending mechanisms and reach their goals. Hence, changing the paradigm and treating this phenomenon through the eyes of the attacker while considering building datasets and transparent intelligent solutions (Maathuis, 2022b) that model and draw relevant principles and requirements for building awareness and defending techniques could be (the basis of) a solution in this direction. Hence, this research aims to build social manipulation security awareness through a deep learning model for generating disinformation tweets. To achieve this objective, multidisciplinary approach is conducted by merging studies from social media, cyber security, information/cyber operations, and deep learning domains through a Design Science Research methodology following a Data Science approach having the following contributions: • Opening a path to scientific and practitioner communities taking offender’s stance when building/enhancing existing social manipulation security awareness through content generation. 227 Proceedings of the 18th International Conference on Cyber Warfare and Security, 2023