Indonesian Journal of Electrical Engineering and Computer Science Vol.26, No.2, May 2022, pp. 826~835 ISSN: 2502-4752, DOI: 10.11591/ijeecs.v26.i2.pp826-835  826 Journal homepage: http://ijeecs.iaescore.com Investigations on spectral efficiency of multi-cell networks using hybrid beamforming Sivaraman Deepa 1 , J. Jeneetha Jebanazer 1 , S. Rajakumar 1 , J. Mercy Sheeba 1 , J. Rryan 2 1 Department of Electronics and Communication Engineering, Panimalar Engineering College, Chennai, India 2 Department of Computer Science and Business Systems, SRM University, Chennai, India Article Info ABSTRACT Article history: Received Dec 19, 2021 Revised Mar 6, 2022 Accepted Mar 16, 2022 Millimeter wave communication systems with antenna beamforming facilitates practical solutions to the capacity crunch issues in the upcoming 5G wireless networks. Multi-cell dense networks are prone to three major interferences-inter-cell, intra-cell and Inter layer interference-the most dominating being the inter-cell interference. This paper focuses to alleviate inter-cell interference using hybrid beamforming (HBF) approach, leveraging coordinated multipoint (CoMP) technique, thereby improving the SE of 5G networks. Simulation results show HBF peforms in par with optimal weights, making it a suitable candidate for 5G networks. As the number of data streams is increased from Ns=1 to 4 for 0 dB signal to noise ratio (SNR) with Nt=64 and Nr=16, the SE increases from 9.5557 bits/s/Hz to 26.423 bits/s/Hz for optimal weights and from 9.1885 bits/s/Hz to 19.763 bits/s/Hz and hybrid weights, respectively. The second set of experiments are conducted to study the effect of number of transmit antennas on spectral efficiency (SE). The results show that as the number of transmit antennas is increased from Nt=16 to 64 for 0 dB SNR, with Nr=16 and Ns=4, the SE increases from 17.735 bits/s/Hz to 26.423 bits/s/Hz and 13.750 bits/s/Hz to 19.763 bits/s/Hz for optimal weights and hybrid weights, respectively. Keywords: CoMP techniques Hybrid beamforming Inter-cell interference Millimeter-wave Multi-cell networks Spectral efficiency This is an open access article under the CC BY-SA license. Corresponding Author: Sivaraman Deepa Department of Electronics and Communication Engineering, Panimalar Engineering College Chennai, India Email: dineshdeepas1977@gmail.com 1. INTRODUCTION The need to address the bandwidth requirements of multitude of users in the upcoming 5G networks, and to solve the capacity crunch issues owing to the ever-increasing wireless user data traffic, has been the root-cause for exploring the spectral region beyond the 6 GHz frequency band. In the recent years, extensive research has been done to investigate the deployment of millimeter-wave systems in 5G networks. The millimeter waves span a wider spectral region from 30 GHz to 300 GHz and occupy the spectral region between microwaves and infrared radiation (IR) waves. The high frequencies incorporated by millimeter waves and their propagation characteristics make them suitable to transmit enormous data and finds its application in cellular-based communications and in radar community. Millimeter-wave systems can achieve humongous data rates and poses practical and reasonable solutions to the afore-mentioned issues of bandwidth shortage and capacity crunch problems. Thus, mmWave communication systems have transpired as the most prominent technology for next generation 5G networks. Spectrum being the most important resource, should be efficiently utilized to ensure that its potential benefits are exploited to the fullest for the social and economic welfare of any nation. The tremendous growth of mobile telecommunications indicates the significance of frequency spectrum. Beamforming using multiple antennas is a key element for effective