Received 20 July 2022, accepted 21 August 2022, date of publication 25 August 2022, date of current version 6 September 2022. Digital Object Identifier 10.1109/ACCESS.2022.3201527 Energy Efficient Resource Allocation for H-NOMA Assisted B5G HetNets UMAR GHAFOOR 1 , HUMAYUN ZUBAIR KHAN 1 , (Senior Member, IEEE), MUDASSAR ALI 1,2 , ADIL MASOOD SIDDIQUI 1 , MUHAMMAD NAEEM 3 , AND IMRAN RASHID 1 1 Department of Electrical Engineering, Military College of Signals, National University of Sciences and Technology, Islamabad 44000, Pakistan 2 Department of Telecommunication Engineering, University of Engineering and Technology, Taxila 47050, Pakistan 3 Department of Electrical and Computer Engineering, COMSATS University Islamabad, Wah Campus, Wah Cantt 47040, Pakistan Corresponding author: Umar Ghafoor (ch.umar163@mcs.edu.pk) ABSTRACT The resource allocation solution offered based on non-orthogonal multiple access (NOMA) and orthogonal multiple access (OMA) schemes are sub-optimal to address the challenging quality of service (QoS) and higher data rate viz-a-viz energy efficiency (EE) requirements in 5th generation (5G) cellular networks. In this work, we maximize the EE using user equipment (UE) clustering (UE-C) with downlink hybrid NOMA (H-NOMA) assisted beyond 5G (B5G) HetNets. We formulate an optimization problem incorporating UE admission in a cluster, UE association with a base station (BS), and power allocation assisted by H-NOMA, i.e., OMA and NOMA schemes in the macro base station (MBS) only and heterogeneous networks (HetNets) environments. The problem formulated is a type of non-linear concave fractional programming (CFP) problem. The Charnes-Cooper transformation (CCT) is applied to the formulated non-linear CFP problem to convert it into a concave optimization, i.e., mixed-integer non- linear programming (MINLP) problem. A two-phase ǫ -optimal outer approximation algorithm (OAA) is used to solve the MINLP problem. The simulation results show that H-NOMA with HetNets outperforms H-NOMA with MBS only in terms of UE admission, UE association, throughput, and EE. INDEX TERMS UE-clustering, H-NOMA, fractional programming, MINLP, energy efficiency. I. INTRODUCTION The rapid growth in the number of mobile user equipment (UE), and heavy data-driven applications, i.e., live video gaming, video streaming, social networking, etc are imposing challenging requirements like minimum delay, higher data rates, spectrum efficiency (SE), and energy efficiency (EE) on the beyond 5th Generation (B5G) cellular networks. This exponential growth in mobile UEs viz-a-viz mobile data traffic is adding to a significant increase in the energy con- sumption in cellular networks. Information communication technology (ICT) is consuming almost 2% of the world’s total energy. Energy consumption emits carbon which causes the greenhouse effect. Thus, ICT offering higher data rates, low carbon emission, and EE are the prime considerations in B5G cellular networks. Thus, academia and industry in The associate editor coordinating the review of this manuscript and approving it for publication was Jie Tang . the wireless communication field must divert their research towards future energy-efficient green cellular networks in B5G networks [1], [2], [3]. Energy-efficient radio resource management techniques are required to satisfy the needs for quality of service (QoS) and higher data rates with minimum energy con- sumption. EE is the ratio of data rate to the total energy consumed [4], [5]. EE can be improved using heterogeneous networks (HetNets), which include a macro base station (MBS) and small base stations (SBSs) [6], [7], [8]. HetNets cover more geographical areas and offer higher data rates and are energy efficient than the conventional MBS-only networks. In Het- Nets, MBS is large with high transmit power, and SBSs (i.e., femtocell, picocell, etc.) are of small size with low transmit power [9]. A low transmit-powered SBS will also preserve energy because additional energy will not be required for cooling purposes compared to a high transmit-powered MBS. VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 91699