O PTIMAL LATENT SPACE FORECASTING FOR LARGE COLLECTIONS OF SHORT TIME SERIES USING T EMPORAL MATRIX FACTORIZATION APREPRINT Himanshi Charotia Mastercard AI Garage Gurgaon, India Abhishek Garg Mastercard AI Garage Gurgaon, India Gaurav Dhama Mastercard AI Garage Gurgaon, India Naman Maheshwari Mastercard AI Garage Gurgaon, India December 16, 2021 ABSTRACT In the context of time series forecasting, it is a common practice to evaluate multiple methods and choose one of these methods or an ensemble for producing the best forecasts. However, choosing among different ensembles over multiple methods remains a challenging task that undergoes a combinatorial explosion as the number of methods increases. In the context of demand forecasting or revenue forecasting, this challenge is further exacerbated by a large number of time series as well as limited historical data points available due to changing business context. Although deep learning forecasting methods aim to simultaneously forecast large collections of time series, they become challenging to apply in such scenarios due to the limited history available and might not yield desirable results. We propose a framework for forecasting short high-dimensional time series data by combining low-rank temporal matrix factorization and optimal model selection on latent time series using cross-validation. We demonstrate that forecasting the latent factors leads to significant performance gains as compared to directly applying different uni-variate models on time series. Performance has been validated on a truncated version of the M4 monthly dataset which contains time series data from multiple domains showing the general applicability of the method. Moreover, it is amenable to incorporating the analyst view of the future owing to the low number of latent factors which is usually impractical when applying forecasting methods directly to high dimensional datasets. Keywords Short length time series Forecasting · Latent representation · Optimal forecast selection · Temporal Factorization 1 Introduction With the easy availability of high-end computing, modern business forecasting applications typically forecast millions of time series depending on the business requirements. These forecasts might be required for different time horizons (monthly, quarterly or yearly) depending on the particular business context and the accuracy of certain time series predictions might matter more than the others (e.g. company revenue predictions for external investor relations vs product level revenues for internal financial planning). One of the most crucial decisions that determine the successful prediction of these series is selecting the appropriate forecasting method which is best suited for each series depending upon its properties (trend, seasonality, etc.). For high-dimensional time series, the analyst typically needs to resort to multivariate methods or design a system that can automatically select the most appropriate uni-variate method. The complexity further adds up when the time series is short in length since historical data beyond a certain point might not be relevant because of constantly changing business scenarios. As observed in many real-life scenarios, there is often a need to forecast demand for products/customers for which no/less history is present. This can arise due to lack of good data collection method (rainfall data may not be present due to infrastructure limitation [1]), a new product launch (every season the substantial amount of new products are launched in fashion industry [2]) or due changing business scenarios(computing the capital gain and profit for risk arXiv:2112.08052v1 [cs.LG] 15 Dec 2021