Gated Memory Unit: A Novel Recurrent Neural Network Architecture for Sequential Analysis Arav Kumar 1 and Gabriel Nasrallah 2 1 Kingswood Oxford, 170 Kingswood Road, West Hartford, CT, USA. 2 Oakland Christian School, 3075 Shimmons Rd, Auburn Hills, MI, USA. Corresponding authors. E-mails: ReplyToAK10@gmail.com; Gabenas8@gmail.com Abstract The predominant models used to analyze sequential data today are re- current neural networks, specifically Long-Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) models, which utilize a temporal value known as the hidden state. These recurrent neural networks process se- quential data by storing and modifying a hidden state through the use of mathematical functions known as gates. However, these networks hold many flaws such as limited temporal vision, insufficient memory capac- ity, and ineffective training times. In response, we propose a simple ar- chitecture, the Gated Memory Unit, which utilizes a new element, the hidden stack, a data stack implementation of the hidden state, as well as novel gates. This, along with a parameterized bounded activation function (PBA), allows the Gated Memory Unit (GMU) to outperform existing re- current models effectively and efficiently. Trials on three datasets were used to display the new architecture’s superior performance and reduced training time as well as the utility of the novel hidden stack compared to existing recurrent networks. On data which measures the daily death rate of SARS-Cov-2, the GMU was able to reduce losses to half that of comparable models and did so in nearly half the training time. Addition- ally, through the use of a generated spiking dataset, the GMU depicted its ability to use its hidden stack to store information past directly observable time steps. We prove that the Gated Memory Unit performs well on a variety of tasks and can outperform existing recurrent architectures. Keywords: Recurrent Neural Network, Gated Recurrent Unit, Gated Memory Unit, Long-Short Term Memory, Sequential Analysis 1