Preprint. Under Review. C ONNECTED H IDDEN N EURONS (CHNN ET ): A N A RTIFICIAL N EURAL N ETWORK FOR R APID C ON - VERGENCE Rafiad Sadat Shahir, Zayed Humayun, Mashrufa Akter Tamim, Shouri Saha Department of Computer Science & Engineering BRAC University Dhaka, Bangladesh {rafiad.sadat.shahir,zayed.humayun,mashrufa.akter.tamim, shouri.saha}@g.bracu.ac.bd Md. Golam Rabiul Alam Department of Computer Science & Engineering BRAC University Dhaka, Bangladesh rabiul.alam@bracu.ac.bd ABSTRACT Despite artificial neural networks being inspired by the functionalities of biolog- ical neural networks, unlike biological neural networks, conventional artificial neural networks are often structured hierarchically, which can impede the flow of information between neurons as the neurons in the same layer have no connec- tions between them. Hence, we propose a more robust model of artificial neural networks where the hidden neurons, residing in the same hidden layer, are inter- connected that leads to rapid convergence. With the experimental study of our proposed model in deep networks, we demonstrate that the model results in a no- ticeable increase in convergence rate compared to the conventional feed-forward neural network. 1 I NTRODUCTION The biological neural networks process large amounts of data passed by senses from different parts of the body (Palm, 1986). A brain can have approximately 100 billion neurons and 100 trillion neural connections, which implies that each neuron can have connections with 1000 other neurons (Glasser et al., 2016). Moreover, the neurons in the brain form complex and dense connections among them- selves, which is important for efficient and flexible information processing (Sporns, 2013). Although the operation of biological neurons served as inspiration for neural networks as they are used in com- puters, many of the designs have since gotten very disconnected from biological reality. (Akomolafe, 2013). Artificial neural networks (ANNs) often follow hierarchical structures with simple neural connections that can impede the flow of information between neurons, as the neurons in the same layer have no connections between them. In some scenarios, to improve the generalization power of new and unseen data, it is important to have more connections among the neurons, as a network with more connections can learn more robust and meaningful features (Zhang et al., 2016). Moreover, having more connections among the neurons can potentially speed up the convergence rate, as it helps to learn complex patterns and relations in the data (Goodfellow et al., 2016). We hypothesize that designing a neural network model with an increased number of neural connections will result in a performance gain in terms of learning. In conventional ANNs, specifically in feed-forward neural networks (FNNs), to increase the number of connections while keeping the number of layers fixed, the number of neurons per hidden layer has to be increased (Goodfellow et al., 2016). However, in- creasing the number of neurons can lead to a slow convergence problem in the model (Gron, 2017). To achieve rapid learning, extensive research has been conducted on various aspects of neural net- work design, e.g. adaptive gradient methods such as the Adam optimizer (Kingma & Ba, 2014), and 1 arXiv:2305.10468v2 [cs.NE] 24 Sep 2023