Q-Learning-based LTE-U and WiFi Coexistence Algorithm for Wireless Healthcare Systems Yuhan Su # , Lianfen Huang, Xiaojiang Du † , Amr Mohamed * , Haotian Chi † , Mohsen Guizani ‡ # Dept. of Communication Engineering, Xiamen University, Xiamen 361005, China † Dept. of Computer and Information Sciences, Temple University, Philadelphia, PA, USA * Dept. of Computer Science and Engineering Department, Qatar University, 2713, Doha, Qatar ‡ Dept. of Electrical and Computer Engineering, University of Idaho, Moscow, Idaho, USA Email: suyuhan066@foxmail.com, lfhuang@xmu.edu.cn, dxj@ieee.org, amrm@qu.edu.qa, htchi@temple.edu, mguizani@ieee.org Abstract— Due to the lack of resources in the low spectrum, LTE-U (LTE in unlicensed spectrum) technology has been proposed to extend LTE to unlicensed spectrum. LTE-U undertakes the task of medical streaming data traffic for licensed spectrum, which can greatly improve the system capacity. However, the introduction of LTE-U technology into wireless healthcare systems also brings coexistence with the current WiFi- based systems. In this paper, an LTE-U coexistence algorithm based on Q-learning is proposed in multi-channel scenarios, the algorithm is based on the idea of LTE-U and WiFi in turn, taking into account the fairness and performance which is vital for medical devices with either LTE-U or WiFi capabilities to work together, to optimize the duty cycle. The simulation results show that the proposed coexistence algorithm based on Q-learning can improve the system throughput while ensuring fairness. Keywords— LTE-U, WiFi, Q-Learning, Reinforcement learning, Coexistence algorithm I. INTRODUCTION With the development of mobile Internet, intelligent devices and mobile applications grow rapidly, people can carry out all kinds of communication connections anytime, anywhere. Faced with such a huge amount of equipment access and business growth, industry and academia are trying to achieve the full use of unlicensed spectrum. They proposed the concept of deploying LTE in unlicensed spectrum [2], known as LTE-U technology. The LTE-U technology extend LTE to the unlicensed spectrum, that is, using the LTE standard protocol to communicate on the unlicensed spectrum, and aggregate licensed spectrum and unlicensed spectrum through the CA technology, the licensed part of the data transmission to the unlicensed spectrum [1]. Introducing LTE into a free, common, unlicensed spectrum will inevitably compete and coexist with other unlicensed communication technologies in the same spectrum. Traditional communication technologies represented by WiFi use unlicensed spectrum for data transmission. In order to realize spectrum sharing, access channels can only be accessed through a competitive way. The LTE design used in the licensed spectrum, the spectrum has absolute control, through the base station on the wireless resource centralized scheduling, so as to obtain higher spectral efficiency. Obviously, if only used the unlicensed spectrum as a new spectrum of LTE, the transmission of LTE-U will cause serious interference to WiFi due to WiFi channel detection and backoff mechanism. Therefore, for the time slot and the scheduling system is completely different from the two systems, need additional design reasonable and fair coexistence in order to ensure that both in the unlicensed spectrum good transmission. At present, the academia have proposed a lot of LTE-U coexistence algorithms. In [2], an LTE-U MAC protocol based on LBT is proposed. The algorithm requires that the LTE-U device be detected at the end of the WiFi transmission frame. In [4], an LBT adaptive algorithm is proposed, which requires LTE-U to be aware of the channel at the edge of the subframe, and to select a new free channel for use. In [5], a fair LBT algorithm is proposed, which combines the total throughput of the system and the fairness factor between LTE- U and WiFi, by allocating the appropriate idle period for WiFi to ensure its transmission. The LTE-U Forum proposed a Carrier sensing and adaptive duty cycle based transmission algorithm (CAST) [6]. Abinader et al. [7] proposed a basic framework of the cooperative coexistence algorithm, which describes the general flow of collaborative coexistence algorithms. In [8], a coexistence algorithm for allocating idle slots by LTE according to a predetermined duty cycle is proposed. In addition, several papers (e.g., [13-19]) have studied related wireless and networking issues. In healthcare, Implementable Medical Devices (IMDs) have been widely used in the last few years. These devices are usually controlled by patients or remotely by a third party (doctors, nurses, etc.) and should be well secured to prevent any kind of threat that can be harmful to patients. For instance, an intruder can listen to an IMD’s radio transmission and can frequently learn private data with insignificant effort from the patient. Attackers can have access to the programming radio, directional receiving wires, and other listening gears of the medical devices. One such review done by Halperin et al. [21] has considered this attack of listening stealthily or eavesdropping, could steal patients’ information. Another attack that was reported where the intruder had the capability to create radio transmissions tended to the IMD, or to replay recorded operations (known as replay attack). A study performed by Rathore et al. [22] showed that with a programmable radio, one could control implantable defibrillator by replaying messages thereby incapacitating modified treatments or conveying a stun planned to initiate a deadly heart attack. There are few solutions which address authentication and authorization issues in wireless medical devices. Biometric based approaches use a unique physiological characteristics of the human body and provide authentication to the authorized users. Though these mechanisms may be secure and lightweight, most schemes fail to accommodate the changes of biometric with respect to time. Key management protocols, another way of authentication, were used to provide authentication to the authorized users using symmetric [21], public key [22] and physiological [23], [24], 25] signals for the generation of keys. However, key management protocols based on symmetric and public key concepts are less reliable and incur extra waiting time for the authentication. Also, once these keys are known, the adversary can take control of the entire system. Copyright IPCO-2017 ISSN 2356-5608 5th International Conference on Control Engineering&Information Technology (CEIT-2017) Proceeding of Engineering and Technology –PET Vol.32 pp. 91-96