International Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-8 Issue-4, November 2019 11492 Published By: Blue Eyes Intelligence Engineering & Sciences Publication Retrieval Number: D4260118419/2019©BEIESP DOI:10.35940/ijrte.D4260.118419 Abstract: Forecasting future price of financial instruments (such as equity, bonds and mutual funds) has become an ongoing effort of financial and capital market industry members. The most current technology is usually applied by high economic scale companies to solve the ambitious and complicated problem. This paper presents optimization solution for a deep learning model in forecasting selected Indonesian mutual funds' Net Asset Value (NAV). There is a well-known issue in determining a deep learning parameters in LSTM network like window timestep and number of neurons to be used in getting the optimal learning from the historical data. This research tries to provide solution by utilizing multi-heuristics optimization approach consists of Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) to determine the best LSTM's network parameters, namely window timesteps and number of neurons. The result shows that from the nine selected mutual funds, PSO outperforms GA in optimizing the LSTM model by giving a lower Root Square Mean Error (RMSE) by 460.84% compared to GA's. However, PSO took a longer execution time by 1.78 times of GA's. This paper also confirms that based on RMSE for both training and evaluation dataset, equity mutual fund's forecasted NAV has the highest RMSE followed by fixed income mutual fund's forecasted NAV and money market mutual fund forecasted NAV. Keywords: long short-term memory; recurrent neural network; genetic algorithm; particle swarm optimization; financial instruments prediction; mutual funds I. INTRODUCTION Financial instruments forecasting has been a fascinating subject ever since its establishment, while the prediction itself is both ambitious and complicated issue, the growing collected data nowadays could be a potential fuel to machine learning approach. It is the act of trying to foretell the future value of financial instruments such as mutual funds traded on a market. The efficient-market hypothesis (EMH) suggests that stock prices as one of the financial instrument reflect all currently available information and any changing price are not based on recent revealed information, so it is unpredictable [1]. A famous random walk down wall street also claimed that stock prices could not be accurately predicted by looking at price history [2], Malkiel argued, stock price are best described by a statistical process called a “random walk” meaning deviations from the central value are Revised Manuscript Received on November 15, 2019 * Correspondence Author Hendri, Master’s degree, computer science, Nusantara University Antoni Wibowo, Associate Professor, Department of Decision Sciences, School of Quantitative Sciences , Universiti Utara Malaysia (UUM). Rina Novita Sari , Bachelor of Engineering, Civil Engineering, University of Indonesia random and unpredictable. Malkiel concluded that paying a professional services to predict market is senseless rather than help. This is supported by the fact that most cases the portfolios managed by professional rarely outperform the market average return after deducted by the professional fees. Others disagree and those with these faiths backed them up with various methods and technologies which as appears enable them to predict future price. A prediction methodology for most financial instruments like stock falls into three common categories, they are fundamental analysis, technical analysis and technological methods. Fundamental analysis is derived on the belief that company requires capital to make further progress and if the company operates well, it should be rewarded with additional capital and increased demand of the company stock thus increase the stock’s value. Technical analysis is more concerned on the trends of the price history which form a time-series analysis. There are various techniques are used such as exponential moving average (EMA), head and shoulders, candle stick patterns and many more [3]. With the recently developed method utilizing the technological advance, machine learning are heavily utilized. An Artificial Neural Network (ANN) has been used and demonstrating its capability of addressing complex problems on several areas. ANN approach may be promising to improve investor’s forecasting ability. Multivariate analytical and techniques using both quantitative and qualitative variables have been repeatedly used [4] to assist the basis of investor stock price expectations, as well as influence investment decision making. Stock price prediction itself is considered as a timeseries data. There are several approaches that normally implemented on timeseries data, one of them is Recurrent Neural Network (RNN), specifically Long Short-Term Memory (LSTM) networks, this network by most is claimed more appropriate for stock prediction as it is a time-series data. According to [5] Recurrent Neural Networks (RNNs), particularly those using Long Short-Term Memory (LSTM) hidden units, are powerful and increasingly popular models for learning from sequence data. The experiment done by Siyuan Liu [6] produce accuracy rate of the single layer LSTM by 0.66 and consequently add another layer, by three-layer LSTM model up to 0.72 for the short period of data. Thus, the more stack layers of LSTM model, the higher accuracy of prediction results, and it was believed as its necessary for LSTM network to be combined with existing clustering techniques to gain significant speed ups in training and testing at minimum loss in Timeseries Forecasting using Long Short-Term Memory Optimized by Multi Heuristics Algorithm Hendri, Rina Novita Sari, Antoni Wibowo