Length of Stay Prediction in Neurosurgery with Russian GPT-3 Language Model Compared to Human Expectations Gleb DANILOV a, 1 , Konstantin KOTIK a , Elena SHEVCHENKO a , Dmitriy USACHEV a , Michael SHIFRIN a , Yulia STRUNINA a , Tatyana TSUKANOVA a , Timur ISHANKULOV a , Vasiliy LUKSHIN a and Alexander POTAPOV a a Laboratory of Biomedical Informatics and Artificial Intelligence, National Medical Research Center for Neurosurgery named after N.N. Burdenko, Moscow, Russian Federation Abstract. Patients, relatives, doctors, and healthcare providers anticipate the evidence-based length of stay (LOS) prediction in neurosurgery. This study aimed to assess the quality of LOS prediction with the GPT3 language model upon the narrative medical records in neurosurgery comparing to doctors' and patients' expectations. We found no significant difference (p = 0.109) between doctors', patients', and model’s predictions with neurosurgeons tending to be more accurate in prognosis. The modern neural network language models demonstrate feasibility in LOS prediction. Keywords. Length of stay, neurosurgery, prediction, neural networks, deep learning, natural language processing 1. Introduction Patients and relatives anticipate the evidence-based risk assessment and length of stay (LOS) prediction in high-tech surgery [1,2]. LOS prognosis can also be utilized in clinical resource management. This paper continues a series of our publications on LOS predicting in neurosurgery based on unstructured textual data [5,6]. The current study aimed to assess the quality of LOS prediction with the GPT3 language model upon the narrative medical records in neurosurgery comparing to doctors' and patients' expectations. 2. Methods Our study consisted of two components: 1) training a neural network language model to predict LOS based on unstructured text data collected retrospectively from electronic health records (EHR); 2) comparing the model predictions with the expectations of 1 Corresponding Author, Gleb Danilov, N.N. Burdenko Neurosurgery Center, 4th Tverskaya-Yamskaya str. 16, Moscow 125047, Russian Federation; E-mail: glebda@yandex.ru. Informatics and Technology in Clinical Care and Public Health J. Mantas et al. (Eds.) © 2022 The authors and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License 4.0 (CC BY-NC 4.0). doi:10.3233/SHTI210882 156