1536-1276 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TWC.2016.2640285, IEEE Transactions on Wireless Communications 1 Reward rate maximization and optimal transmission policy of EH device with temporal death in EH-WSNs Shengda Tang and Liansheng Tan Abstract—For the perpetual wireless sensor network (WSN), energy harvesting (EH) technology is emerging as a promising solution. However, the randomness and instability of the har- vested energy may lead to the occurrence of the temporal death, which is harmful to the functions of the WSN, and has negative impact on the Quality of Service (QoS) of the network. With temporal death being taken into account, this paper proposes a novel and overall framework, namely a multi-layer Markov fluid queue (MLMFQ) model, for modeling and analyzing the data transmission nature of the EH devices (EHDs). We formulate the model of the whole system in terms of MLMFQ, and obtain the stead-state probabilities of the EHD. We study the issue of how to maximize the steady-state average reward rate of the reported data packets. We then propose an optimization model, in which one can maximize the overall steady-state average reward rate with relation to transmission policy under the specific constraints. On this basis we are able to propose an algorithm for calculating the optimal transmission policy of the EHD with temporal death. We validate our results using a numerical example and report some interesting findings from our numerical studies. Index Terms—Energy harvesting wireless sensor network (EH- WSN), multi-layer Markov fluid queue (MLMFQ), transmission policy, temporal death, reward rate. I. I NTRODUCTION W Ireless sensor networks (WSNs) are widely used in monitoring and collecting information. A crucial prob- lem in WSNs is the limited energy supply, which unfor- tunately holds back their applications. Fortunately, energy harvesting (EH) makes WSNs possible to exploit energy from the different ambient sources, such as solar power, wind, mechanical vibrations, temperature variations, magnetic fields, etc. Thus WSNs with EH technique have the ability to operate autonomously for ever. The widespread adoption of EH will reduce the use of conventional energy and the accompanying carbon footprint, and will subsequently benefit the environment. In addition, they do not require conventional recharging, enable mobility, and they can be deployed in hard- This work was supported by the National Natural Science Foundation of China under Grant 61370107 and Grant 61672258, and Program on the high level innovation team and outstanding scholars in Universities of Guangxi Province. Shengda Tang is with Department of Computer Science, Central China Normal University, Wuhan, Hubei, PR. China and also with the School of Mathematics and Statistics, Guangxi Normal University, Guilin, Guangxi, P.R. China (email: tangsd911@163.com). Liansheng Tan is with the Computer Science Department of Cen- tral China Normal University, Wuhan 430079, PR. China (email: l.tan@mail.ccnu.edu.cn). Corresponding to Dr. Liansheng Tan. to-reach places such as remote rural areas, within structures, and even within the human body. Being sharply different from the traditional sensor nodes, where one is forced to minimize energy consumption under the performance constraint, if we use wireless sensor nodes that are powered by EH technology, energy conservation is no longer the main concern and the main objective becomes the efficient management of the energy that the nodes can extract from the environment. Recent years have seen works that are focused on develop- ing efficient transmission and resource allocation algorithms with different objectives and energy recharging models. We summarize the relevant publications as follows. In [1], a unified model for energy harvesting nodes in a WSN was developed, and the probability of event (packet) loss due to energy running out was derived therein. In [2], the authors presented a Markov based model for predicting the probability of energy depletion with very low computational complexity. The authors in [3] presented discrete time Markov chain for environmentally powered sensor nodes for assessing statisti- cal properties such as the probability of achieving a given operation time and the expected downtime. In [4], a semi- Markov model was proposed, and the authors derived the residual energy distribution and lifetime prediction of wireless sensor nodes. In the paper [5], the performance was evaluated under various sleeping and waking up strategies with relation to channel state, battery state and environmental factors. In [6], the authors presented a model for characterizing environmental sources, where a power management scheme had provided to ensure the sustainable performance level from a particular source. The authors in [7] considered an energy replenishable sensor in a continuous-time system, using policy iteration to maximize the average value of the reported data. In [8] and [9], the authors studied heuristic delay-minimizing policies and sufficient stability conditions for an EH system with a data queue. In [10], [11], [12] and [13], the authors modeled a slotted time system by using a two-dimensional discrete time stochastic process. They also associate an achievable reward that corresponds to the channel gain in that time slot, with the main goal of maximizing the steady-state average transmission rate. In [14], the authors developed a new framework enabling an adaptive duty cycling scheme for sensor networks that takes into account the node battery level, ambient energy, and application-level Quality of Service (QoS) requirements. They modeled the system as a Markov Decision Process that modifies its state transition policy using reinforcement