IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 58, NO. 4, AUGUST 2011 1793 Switched Ethernet in Synchronized Distributed Control Systems Using RTnet Luca Boncagni, Antonio Barbalace, Yahya Sadeghi, Marco Pompei, Luca Zaccarian, and Filippo Sartori Abstract—We adopt a model of a real-time distributed network where the synchronization is done by an external timing network that triggers the acquisitions. A Switched Ethernet is used to send the synchronously sampled data from nodes to nodes to perform the distributed control algorithm. In order to guarantee low laten- cies and jitter, the traffic on the Switched Ethernet must be care- fully analyzed to be able to correctly model closed-loop systems and potentially identify events that might lead to large delays or even packet losses. In this paper we discuss the most relevant results from a testbed control network with synchronous data acquisition nodes. A switched Gigabit Ethernet is adopted and each node uses RTnet running in a RTAI/Linux Operating System. To resolve the problem of timing uncertainties at the data link layer, we propose a new custom solution based on a TDMA discipline. Index Terms—Gigabit ethernet, linux, plasma fusion, real-time, RTAI, RTnet, switched ethernet, synchronous ethernet, timing net- work. I. INTRODUCTION F EEDBACK distributed control systems in Fusion exper- iments rely on a variety of network technologies. Some sites have adopted high performance computing solutions, as Myrinet [1] in PCS at DIII-D and EAST [2], [3], Reflective Memory at JT-60 and KSTAR [4], [5] or telecom like solutions as ATM at JET [6]. Just few sites have chosen Ethernet for the real-time data transfers, as RFX [7] and FTU [8], in which per- formance studies have been carried out [9], [10]. Switched Ethernet is becoming a valid candidate for dis- tributed control loops systems in plasma fusion devices, potentially replacing more expensive or custom solutions. A networked control system usually has hard real-time require- ments and transmission delays must be taken into account in the overall closed loop design. This implies some modifica- tions from the standard way of adopting Ethernet. Among the Manuscript received June 28, 2010; revised March 12, 2011, April 15, 2011, and April 27, 2011; accepted May 05, 2011. Date of publication June 20, 2011; date of current version August 17, 2011. This work was supported by the Euro- pean Communities within the framework of the ENEA-Euratom Association on Fusion Research. The views and opinions expressed herein do not necessarily reflect those of the European Commission. L. Boncagni is with the Associazione Euratom-ENEA sulla Fusione, Frascati Research Centre, Division of Fusion Physics, Rome 00133, Italy (e-mail: luca. boncagni@enea.it). A. Barbalace is with the Consorzio RFX, Associazione Euratom-ENEA sulla Fusione, 35127 Padova, Italy (e-mail: antonio.barbalace@igi.cnr.it). Y. Sadeghi, M. Pompei and L. Zaccarian are with the Department of Com- puter Science, Systems and Production, University of Rome “Tor Vergata,” 00133 Rome, Italy. F. Sartori is with the Fusion for Energy CODAC, 08019 Barcelona, Spain. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNS.2011.2156427 possible modifications, RTnet is a hard real-time network stack for RTAI/Linux that re-implements UDP/IP in a reduced and optimized fashion. RTnet enables an Ethernet network to be used as a real-time communication media. It has been known, especially in the past, that Ethernet was not suitable for industrial networking because the medium access control (MAC) of Ethernet is the contention-based car- rier-sensing multiple access collision detection (CSMA/CD), that exhibits unstable performance under heavy traffic and unbounded delayes. Switched Ethernet is then a very promising perspective for industrial networking because the switching technology can eliminate frame collisions. Since the colli- sion-less Ethernet is no longer unstable under heavy traffic and its delay can be drastically reduced, the adoption of Switched Ethernet as industrial network is seriously considered along with the use of inexpensive switches. This paper presents the performance evaluation of the Gigabit Switched Ethernet used with the PCI and PCIe databus, with the aim of using it as a communication network for interconnecting various components of a real-time control system. The perfor- mance of the Switched Gigabit Ethernet is experimentally eval- uated on a network testbed. Moreover, an alternative approach to synchronous networking systems is illustrated. This paper is organized as follows. In the Sections II–IV the experimental framework and its components are described; in the Sections V–VI the results of the tests are reported and thoroughly discussed, while in Section VII a customized RTnet TDMA module is illustrated. Sections VIII and IX are devoted to future work and conclusions. II. SWITCHED ETHERNET Ethernet, IEEE802.3 was developed in the early 1970 and is nowadays the basis for the physical and data link layer of In- ternet at least for office communication. Today the specifica- tions include 1 Gb and 10 Gb Ethernet, as well as the former 10BASE-T and 100BASE-T. In order to transmit a frame, a source station checks if the transmission medium is busy. In this case, the source holds the transmission and waits until the medium is idle. Otherwise, if the medium is idle, the source transmits the frame. If a collision is detected during the trans- mission, the source interrupts the transmission and broadcasts a jam signal to notify an occurrence of the collision to the other stations. After the collision of a frame, the source station waits for the backoff, and then retries repeatedly the above trans- mission procedure. If the number of retries exceeds a prede- fined level, the transmission of the corresponding frame is with- drawn. Such an algorithm, CSDMA/CD, makes the communi- 0018-9499/$26.00 © 2011 IEEE