On-Line Adaptive Algorithms in Autonomic Restart Control Hiroyuki Okamura † , Tadashi Dohi † , and Kishor S. Trivedi ‡ † Department of Information Engineering, Graduate School of Engineering Hiroshima University, Higashi-Hiroshima, 739–8527, Japan {okamu, dohi}@rel.hiroshima-u.ac.jp ‡ Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708-0291, USA kst@ee.duke.edu Abstract. Restarts or retries are typical control schemes to meet a deadline in real-time systems, and are regarded as significant environ- mental diversity techniques in dependable computing. This paper recon- siders a restart control studied by van Moorsel and Wolter (2006), and refines their result from theoretical and statistical points of views. Based on the optimality principle, we show that the time-fixed restart time is best even in non-stationary control setting under the assumption of unbounded restart opportunities. Next we study statistical inference for the restart time interval and develop on-line adaptive algorithms for es- timating the optimal restart time interval via non-parametric estimation and reinforcement learning. Finally, these algorithms are compared in a simulation study. Keywords: autonomic restart, on-line adaptive control, non-parametric statis- tics, job completion time, Q-learning. 1 Introduction Since many software systems executed continuously for long periods of time, it is quite important to diversify their operating environments by monitor- ing/controlling them at judiciuosly chosen time epochs. For instance, some faults may cause the software to age due to error conditions that accrue with time and/or load. This phenomenon is called software aging which affects the perfor- mance of applications and eventually cause them to fail [26], and has been ob- served in widely-used communication software like Internet Explorer, Netscape and xrn as well as commercial operating systems and middleware. A comple- mentary approach to handle software aging and its related transient failures, called software rejuvenation, has already become much popular as a typical and low cost environment diversity technique of operational software [10]. In general, software rejuvenation is a preventive and proactive solution that is particularly useful for counteracting the phenomenon of software aging. It involves stopping