Optimal Beamforming in Cognitive Two-way Relay Networks Ardalan Alizadeh, Seyed Mohammad-Sajad Sadough and Navid Tafaghodi Khajavi Cognitive Telecommunication Research Group, Department of Electrical Engineering, Faculty of Electrical and Computer Engineering, Shahid Beheshti University G.C., Evin 1983963113, Tehran, IRAN. Email: ar.alizadeh@mail.sbu.ac.ir, s sadough@sbu.ac.ir, n.tafaghodi@mail.sbu.ac.ir Abstract—We consider a cognitive two-way relay network which consists of two transceivers (primary users) and mul- tiple cognitive terminals. The two transceivers transmit their data toward the cognitive terminals which sense the spectrum constantly. If the primary users are in operation, the cognitive terminals act as relay nodes by multiplying the received signal by a beamforming coefficients and then broadcasting the so- obtained signal to the transceivers. When the primary users are not in operation, the cognitive terminals communicate with a base station (BS) to increase the overall throughputs. Our aim is to optimally calculate the beamforming weight coefficients so as to minimize the total transmit power of the two transceivers and the cognitive radios subject to three constraints on the received signal-to-noise-ratios at the two transceivers and at the cognitive BS. Simulation results show that the sum-rate of the cognitive two-way relay network is increased while the total transmit power is minimized. I. I NTRODUCTION Cognitive radio (CR) [1] is a promising solution to the problem of scarce spectrum resources in wireless commu- nications. The basic idea of CR is to allow a secondary (unlicensed) user to utilize a frequency band already allocated to primary users (PU). The CR has to sense constantly the spectrum to detect the spectral holes before transmitting their own signal. Obviously, they should give back the spectrum once the presence of the PU is detected in order to minimize harmful interference to the licensed users. Cooperative wireless networks have recently been the focus of many research activities [2], [3], [4]. In these schemes, differ- ent users act as relay nodes and collaborate with each other to provide a reliable communication link between a transmitter and a receiver. Typically, the transmission is composed of two steps. In a first step, the transmitter sends its symbols to the relays. Each relay processes its received signal based on its relaying scheme. In a second step, the so-produced signal is transmitted to one or multiple receiver. Recently, decentralized beamforming have been proposed for relaying schemes with the aim of designing optimal beamforming coefficients with respect to a particular criteria. For instance in [5], considering a two-way relaying, the authors find the beamforming vector by minimizing the total power of the network or by maximizing the signal-to-noise ratio (SNR) under appropriate system constraints. In a two-way relaying (also called bi-directional relaying), the relays cooperate with Fig. 1. Architecture of the considered cognitive two-way relay network. each other to establish a connection between two transceivers. In this paper, we consider a two-way relaying scheme where the relay nodes are cognitive terminals. The data processed and broadcasted from the CRs to the transceivers, is also received by the cognitive base station which uses the received signal to sense constantly the spectrum used by the transceivers. This scenario, overcomes the hidden terminal problem which hap- pens when the CR is shadowed or in severe multipath fading [6], [7]. Once the cognitive BS senses that primary transceivers are not in operation, cognitive terminals can opportunistically establish a connection with their own base station (BS) over the same frequency band allocated to primary transceivers. In fact, here, we provide the cooperative relay network with a cognitive feature for increasing the total network throughputs. Our goal is to optimally derive the beamforming coefficients so as to minimize the total power dissipated in the network subject to constraints on the transceivers and the cognitive base station SNRs. In this way, we satisfy a target quality of service (QoS) for the transceivers and we increase the detection probability of cognitive spectrum sensing. II. TRANSMISSION MODEL As shown in Fig. 1, our network is composed of two primary transceivers, n r decentralized CR terminals or relay nodes between the two transceivers and one cognitive BS (also referred to as fusion center). We assume that there is no direct link between the two transceivers and thus, they have to communicate by means of the CR terminals. In this scenario, the CR network uses 21st Annual IEEE International Symposium on Personal, Indoor and Mobile Radio Communications 978-1-4244-8015-9/10/$26.00 ©2010 IEEE 2329