The GENI Experiment Engine Andy Bavier Jim Chen Joe Mambretti Rick McGeer PlanetWorks, LLC and iCAIR, iCAIR, Communication Design Group Princeton University Northwestern University Northwestern University US Ignite, USA acb@cs.princeton.edu jim-chen@northwestern.edu j-mambretti@northwestern.edu rick.mcgeer@us-ignite.org Sean McGeer Jude Nelson Patrick O’Connell Glenn Ricart Duck Syrup Princeton University Duck Syrup US Ignite seanmcgeer@gmail.com jcnelson@cs.princeton.edu patrickboconnell@gmail.com glenn.ricart@us-ignite.org Stephen Tredger Yvonne Coady University of Victoria University of Victoria stredger@gmail.com ycoady@cs.uvic.ca ABSTRACT We describe the GENI Experiment Engine, a Distributed- Platform-as-a-Service facility designed to be implemented on a distributed testbed or infrastructure. The GEE is in- tended to provide rapid and convenient access to a dis- tributed infrastructure for simple, easy-to-configure exper- iments and applications. Specifically, the design goal of the GEE is to permit experimenters and application writers to: (a) allocate a GEE slicelet; (b) deploy a simple experiment or application; (c) run the experiment; (d) collect the results; and (e) tear down the experiment, starting from scratch, within five minutes. The GEE consists of a set of cooper- ating services over the GENI infrastructure, which together with rapidly-allocated slicelets and a rapidly-allocated net- work offers a complete, ready to use, sliceable platform over the GENI Infrastructure. The GEE is designed to use off-the-shelf components and infrastructure; unlike previous PaaS offerings, it can be nested nicely inside a GENI slice, or any other IaaS infrastructure. Further, the GEE’s south- bound interface is extremely small and lightweight, making it portable to other underlying infrastructures. 1. INTRODUCTION AND MOTIVATION Over the course of the past few years, we have partici- pated in the development of a number of demonstrations of the Global Environment for Network Innovateion (GENI) infrastructure [7] , including a distributed geo- graphic information system, a distributed Map/Reduce system over the wide area, and a distributed media transcoding system. Many of these were built on our earlier TransCloud system [5]. The first and most com- mon question we got after each demo was: “Can I use the infrastructure you built on top of GENI?” This ques- tion was understandable: GENI is a distributed, highly- configurable Infrastructure-as-a-Service (IaaS) Cloud with deeply-programmable networking, designed to per- mit any experimenter to construct his own Internet on the GENI substrate. This platform offers great power and flexibility to its users, experimenters, and applica- tion developers, but at a price: allocating and configur- ing a GENI slice is a far more cumbersome task than the relatively lightweight mechanisms that characterize PlanetLab and other Cloud infrastructures that offer less freedom to users than GENI does. All we needed was a way to deploy VMs across the wide area, and on GENI we found deployment and maintenance of our demonstrations to be a significant challenge. Further, all our demonstrations had the same essential compo- nents: a network of virtual machines (or, more pre- cisely, some platform isolated from others in a multi- tenant environment); some form of wide-area messag- ing system; a conceptually- (and, generally, physically- ) distributed store; an orchestration system, typically a bunch of Python or Perl scripts which invoked ssh com- mands on the various nodes. We were building on top of what was essentially PlanetLab [25] plus a few rela- tively standardized services. It would be convenient if a permanent infrastructure were available that had those services. The flexibility of the underlying GENI infrastructure permits users to allocate expensive resources which are in short supply, primarily physical machines and heavy- weight virtual machines. Since expensive resources can- not be allocated for long periods of time, lease times for GENI slices are short, requiring frequent slice renewal. Given the range of experiments and applications target- ted by GENI, this is a requirement for the underlying infrastructure: one can’t design an infrastucture antic- ipating long-duration use of lightweight resources and easily accomodate short-term requests for expensive re- sources. It is very hard to turn part of an aparment building into a mansion. However, many GENI exper- iments and applications don’t require these expensive resources. For these users, the full flexibility and free- dom GENI offers little benefit, and the machinery this freedom requires imposes nontrivial burdens. 1 TRIDENTCOM 2015, June 24-25, Vancouver, Canada Copyright © 2015 ICST DOI 10.4108/icst.tridentcom.2015.259974