INTEGRATION OF SUSTAINABLE ENERGY CONFERENCE (ISENEC), NUREMBERG, 2016 (PREPRINT) Dynamic Programming for Smart Water Heater Control Dominik Meyer, Christoph Passenberg, Johannes Feldmaier, Hao Shen Chair for Data Processing, Technical University of Munich {dominik.meyer, christophpassenberg, johannes.feldmaier, hao.shen}@tum.de I. ABSTRACT The modern energy grid encompasses more and more re- newable energy sources to be integrated in the grid. Those renewable energy sources not only consist of big managed wind- or solarfarms, but also of solar generators on private homes. These smart homes have a comparatively small size and are therefore not much regulated with respect to feeding power back into the grid. For the homeowner on the other hand it is also un- favourable to feed all generated electricy back to the grid as the prices for selling are most of the times lower than for buying. Eigen consumption (direct consumption of gener- ated electricy in the house) is therefore the best option in most cases, cf. [1]. Since the electicity production during the day and consumption, occuring mostly during early morning or late evening hours does not match up, an energy storage solution could ease this mismatch. More and more batteries, to store a certain amount of solar electicity are used in private homes, but are still expensive and difficult to manage. On the other hand, every household has a hot water tank installed which could be easily retrofitted with an additional electrical heater to store the excess solar energy in thermal form. Unfortunately, compared to batteries, this energy cannot be easily converted back to electicity. It has therefore be taken careful consideration when and how much energy to put into the hot water tank. Additionally, various uncertainties, such as weather, rule the energy generation process as well as the hot water and electricity consumption by the household residents. We therefore employ a stochastic dynamic programming algorithm to find the optimal control schedule for the hot water tank to ensure enough hot water at all times, while maximizing the eigen consumption of solar power. This in effect, results in a reduction externally pruchased energy. Forecasts warm water consumption electrical demand weather data energy price Smart Water Heater Control Heated Tank Storage Fig. 1. Information flow in the system. The stochastic dynamic programming [2] algorithm seeks to find an optimal policy. This means, that in each possible system state, the algorithm determines the optimal action, that leads to a reduced cost while obeying the system constraints. In order to proactively act on future hot water consumption or weather change, we utilize forecasts for those variables, compare Figure 1. They are adaptively considered in our algorithm utilizing the information about the prediction uncertainty of the forecast algorithm. In contrast to other adaptive systems, as e.g. neural net- works [3], our method only needs a one-time parametriza- tion for the household parameters it seeks to control and no tedious parameter tuning process is necessary. In the simulations, the algorithm demonstrated its capability to realiably ensure the hot water supply despite the uncertainty in the hot water consumption prediction. Furthermore, promising results show the feasibility to reduce the energy costs at the same time, both in a constant and real time energy pricing scheme. Additinally, other means of energy production, such as solar powered water heating or gas fired hot water tanks can be easily integrated in the system. II. REFERENCES [1] Q. Wei, D. Liu, G. Shi, Y. Liu, and Q. Guan, ”Optimal self-learning battery control in smart residential grids by iterative q-learning algorithm,” in Adaptive Dy- namic Programming and Reinforcement Learning, IEEE Symposium on, Dec 2014, pp. 1–7. [2] D. P. Bertsekas, Dynamic programming and optimal control. Athena Scientific Belmont, MA, 1995. [3] D. Fuselli, F. De Angelis, M. Boaro, D. Liu, Q. Wei, S. Squartini, and F. Piazza, ”Optimal battery management with adhdp in smart home environments,” in Advances in Neural Networks. Springer Berlin Heidelberg, 2012, vol. 7368, pp. 355–364.