Contents lists available at ScienceDirect Electric Power Systems Research journal homepage: www.elsevier.com/locate/epsr Incentive-based load shifting dynamics and aggregators response predictability Pedro M.S. Carvalho a, ⁎ , José D.S. Peres a , Luís A.F.M. Ferreira a , Marija D. Ilic b , Michelle Lauer b , Rupamathi Jaddivada b a IST and INESC ID, Universidade de Lisboa, Lisbon, Portugal b Massachusetts Institute of Technology, Cambridge, USA ARTICLE INFO Keywords: Demand response Machine learning Dynamic systems Ancillary services ABSTRACT Demand fexibility and its responsiveness under price-based control is a major research feld. Much attention has been paid to model demand elasticity as synonymous with demand fexibility. But demand fexibility comes mostly as load shifting, and load shifting dynamics have been neglected when modelling demand response. In this paper, we model load shifting dynamics to simulate aggregate responses and analyse their predictability under time-varying prices. Our experience with simulation is then used to discuss possible enhancements in machine learning capable of predicting aggregate load dynamics. 1. Introduction Demand response is seen as a major potential contributor to support the increased penetration of renewables. Demand may be seen as an additional resource of fexibility and contribute in a similar way as the supply side resources to provide ancillary services [1]. In order to qualify demand response as a valid contributor, insight onto load-fol- lowing capabilities and its predictability needs to be gained. An impressive bibliography has recently become available in the feld of demand response. Various resource types have been studied, from electric vehicles (EV) to water heaters and air-conditioners [2, 3] and diferent strategies for control have been addressed, from price- based [4, 5] to direct control [6, 7]. Some papers propose centralized approaches that rely on information on load use to optimize customer responses. Other papers propose decentralized approaches that rely on load forecast to assess response elasticity w.r.t. varying tarifs [8]. Most papers focus on peak-load reduction only. Few papers deal with load- following. Those that deal with load-following take demand response as the bid-quantity market ofers often ignoring the dynamic specifc limitations of load response [9, 10]. Very few works address the dy- namics of load response and the limitations imposed by them [11]. We have recently advanced formal hypotheses on the ramping limitations of load-shifting to conclude about the underlying potential of demand response [12]. Such limitations were derived for ideal direct control over hidden shifting constraints, as enabled by future ICT and imposed by load use confdentiality, respectively. Under ideal direct control, load predictability is not an issue: within the intrinsic limitations of shifting, load responds as expected. Yet, under indirect control enabled by time-varying prices or incentives, response predictability becomes very challenging. Without real-time auction mechanisms, indirect control based on prices introduces an uncertainty factor into the market clearing process, which comes from the underlying uncertainty in user availability to respond. From the market perspective, this works like a supply uncertainty factor. Uncertainty tends to attenuate with the size of the population of demand users, improving predictability in relative terms. There are however difculties associated with efects that do not attenuate with size, such as those related to response dynamics. Besides the underlying uncertainty in availability, indirect control introduces a dynamical ef- fect into the market clearing process: when a load use is shifted ahead in a time period k, this same use will probably be available to be shifted again in + k 1 for a similar price. For a given load-change demand, the response dependents much on the past responses rather than just on the bidding curve. This results from the supply dynamics induced by the load adjustments. From the market perspective, this works like a dy- namic supply factor. We recently explored several machine learning methods for gen- erating accurate household energy usage predictions [13]. A real-time iterative auction mechanism to assess the controllability of the house- hold devices in response to accurately anticipated prices has been as- sessed in [14–17]. Such a method assumes prices are accurate and proved to be capable of learning load-use dynamics but cannot translate https://doi.org/10.1016/j.epsr.2020.106744 Received 4 October 2019; Received in revised form 9 April 2020; Accepted 2 August 2020 ⁎ Corresponding author. E-mail address: pcarvalho@ist.utl.pt (P.M.S. Carvalho). Electric Power Systems Research 189 (2020) 106744 Available online 10 August 2020 0378-7796/ © 2020 Elsevier B.V. All rights reserved. T