Sponsored by the National Science Foundation LabWiki Tutorial (OMF/OML) Divya Bhat, Mike Zink, Pieter Becue, Brecht Vermeulen GEC20 July 8 th 2014, Ghent, Belgium

Sponsored by the National Science Foundation 2 GEC20 Experiment Design “Perform basic measurements on GENI testbeds” GENI slice (network & compute resources) Repeatable experiment Ability to monitor and document experiment

Sponsored by the National Science Foundation 3 GEC20 The “Experiment Cycle” Prepare Plan Execute Notes Code Experiment Description Runs Measurements

Sponsored by the National Science Foundation 4 GEC20 Prepare Plan Execute Notes Code Experiment Description Runs Measurements Publish The “Successful Experiment Cycle”

Sponsored by the National Science Foundation 5 GEC20 The “Experiment Cycle” in a Tool: LabWiki

Sponsored by the National Science Foundation 6 GEC20 WikiEditRun Search

Sponsored by the National Science Foundation 7 GEC20 LabWiki Core Plugin Your Plugin GENI/FIRE CH/AM GENI/FIRE CH/AM OMF PostGres Your Service Your Service LabWiki Services PlanPrepareExecute

Sponsored by the National Science Foundation 8 GEC20 Learning Switch A A’A’ B B’B’C C’C’ A A’ Source: A Dest: A’ MAC addr interface TTL Switch table (initially empty) A 1 60 A A’ frame destination unknown: flood A’ A  destination A location known: A’A’ 4 60 selective send  More info in chapter 6 of “Computer Networks”, Kurose & Ross

Sponsored by the National Science Foundation 9 GEC20 Example Experiment Topology N1 – N6: regular end systems Switch1,Switch2: VMs with OVS installed  sw-based OF switch Trema-based OF controller running on Switch Goal: Implement learning switch and verify its correct behavior

Sponsored by the National Science Foundation 10 GEC20 LabWiki Tour

Sponsored by the National Science Foundation 11 GEC20 GENI TestBed OML Server LabWiki PostGres 1.Instrument 2.Run 3.Collect 4.Plot 5.Save OML Client 6.Obtain Environment 0.Reserve Experimenter 2.Run 3.Collect 4.Plot 5.Save

Sponsored by the National Science Foundation 12 GEC20 Preparation In earlier tutorials: –GENI/FIRE account and credentials –Know what resources are available –How to create a slice request (Rspec) –How to reserve a slice –How to log into VMs of your slice Familiar with GENI Portal!

Sponsored by the National Science Foundation 13 GEC20 Part 1 E/Labwiki/Part1/Introduction

Sponsored by the National Science Foundation 14 GEC20 Your Slice Architecture Persistent Server OML Server Job Service AMQP Server EC PostGres RCRC RCRC MLML MLML RC ML RCRC RCRC MLML MLML RCRC RCRC MLML MLML RCRC RCRC MLML MLML LabWiki RC ML RCRC RCRC MLML MLML RCRC RCRC MLML MLML

Sponsored by the National Science Foundation 15 GEC20 Persistent Server OML Server AMQP Server EC PostGres LabWiki 15 RCRC RCRC MLML MLML RCRC RCRC MLML MLML RCRC RCRC MLML MLML AMQP Messaging Images with tools available for InstaGENI, ExoGENI and iMinds Job Service

Sponsored by the National Science Foundation 16 GEC20 Persistent Server OML Server AMQP Server EC iRODS LabWiki 16 RCRC RCRC MLML MLML RCRC RCRC MLML MLML RCRC RCRC MLML MLML EC/RC Messaging Job Service

Sponsored by the National Science Foundation 17 GEC20 Persistent Server OML Server AMQP Server EC PostGres LabWiki 17 RCRC RCRC MLML MLML RCRC RCRC MLML MLML RCRC RCRC MLML MLML Data Collection Job Service

Sponsored by the National Science Foundation 18 GEC20 Persistent Server OML Server AMQP Server EC PostGres LabWiki 18 RCRC RCRC MLML MLML RCRC RCRC MLML MLML RCRC RCRC MLML MLML LabWiki Job Service

Sponsored by the National Science Foundation 19 GEC20 Documentation OML: Job Service: OMF: AMQP: LabWiki: OVS: Trema:

Sponsored by the National Science Foundation 20 GEC20 Steps 1.Reserve slice with VMs that boot LabWiki image 2.Verify that slice is up 3.Define experiment to be executed in LabWiki 4.Run and observe experiment 5.Archive results 6.Document results 7.Delete slice (be a good citizen by sharing!!)

Sponsored by the National Science Foundation 21 GEC20 Nodes 1-6 Image Postboot script ssh into node

Sponsored by the National Science Foundation 22 GEC20 Nodes 1-6: Image InstaGENI –Ubuntu –OMF –OML client & apps –URL to image: hp?uuid=df9ce62b-ec1a-11e3-bc1a

Sponsored by the National Science Foundation 23 GEC20 Nodes 1-6: Postboot Script IG Postboot script ( testbeds.ilabt.iminds.be/FGRE/fgre_wall1.sh) #!/bin/bash read -r slice1 </var/emulab/boot/nickname slicename=$(echo $slice1 | cut -f2 -d.) host1=$(hostname) host=$(echo $host1 | cut -f1 -d.) slice=`ruby -e "print '$slicename'[/[^+]*$/]"` echo $host > /etc/hostname /bin/hostname -F /etc/hostname echo "--- :uid: $host-$slice :uri: amqp://gimi3.casa.umass.edu :environment: production :debug: false" > /etc/omf_rc/config.yml restart omf_rc Initiates connect to AMQP server

Sponsored by the National Science Foundation 24 GEC20 Switch1 & Switch2 Image Postboot script ssh into node

Sponsored by the National Science Foundation 25 GEC20 Switch1 & Switch2: Image InstaGENI –Ubuntu –OMF –OML and apps –OVS –Trema –URL to image: p?uuid=551210c7-ecca-11e

Sponsored by the National Science Foundation 26 GEC20 Switch: Postboot Script Postboot script: Rspec:

Sponsored by the National Science Foundation 27 GEC20 Switch: OVS Postboot sudo ovs-vsctl list-br | grep -q $SWITCHNAME if [ $? -ne 0 ]; then sudo ovs-vsctl add-br $SWITCHNAME sudo ovs-vsctl set-fail-mode $SWITCHNAME secure fi # Create ports for each of the interfaces if they don't exist # and clear the IP address for i in $IFLIST; do sudo ovs-vsctl list-ports $SWITCHNAME | grep -q ${i} if [ $? -ne 0 ]; then sudo ovs-vsctl add-port $SWITCHNAME ${i} sudo ifconfig ${i} fi done ovs-vsctl set-controller test tcp: :6653 /usr/bin/trema run /root/learning-switch_copy.rb > /tmp/trema-runlog 2>&1 &

Sponsored by the National Science Foundation 28 GEC20 Debugging the RC ssh into node Click on ‘Login’ –Experiment Controller -> Raw Rspec -> Login On the node: /var/log/omf_rc.log [ :49: ] WARN OmfCommon::Comm::AMQP::Communicator: Lost connectivity. Trying to reconnect... [ :49: ] INFO OmfRc::Runner: Starting OMF Resource Controller version '6.1.2.pre.5' [ :49: ] INFO OmfRc::Runner: Connected using {:proto=>:amqp, :user=>"guest", :domain=>" "} [ :56: ] INFO OmfRc::ResourceProxy::AbstractResource: App Event from 'ping_cxt_0' (#0) - STARTED: 'env -i /usr/local/bin/ping-oml2 -a c 30 --oml-config /tmp/854849b2-19a2-475d d894aa xml' [ :56: ] INFO OmfRc::ResourceProxy::AbstractResource: App Event from 'ping_cxt_1' (#0) - STARTED: 'env -i /usr/local/bin/ping-oml2 -a c 30 --oml-config /tmp/96c56027-f62e-41b1-873d- ba39fe429ab xml' [ :56: ] INFO OmfRc::ResourceProxy::AbstractResource: App Event from 'ping_cxt_2' (#0) - STARTED: 'env -i /usr/local/bin/ping-oml2 -a c 30 --oml-config /tmp/53a d6-4b31-8f65- 4e272d7b xml'

Sponsored by the National Science Foundation 29 GEC20 OEDL Script We will look at this in LabWiki More info on OEDL scripts: OMF6

Sponsored by the National Science Foundation 30 GEC20 MD Script We will look at this in LabWiki

Sponsored by the National Science Foundation 31 GEC20 Part 2 – Add Flow Statistics RE/Labwiki/Part2a/Introduction

Sponsored by the National Science Foundation 32 GEC20 Measure FlowStats Observe performance of flows Done via flow_stats in OF controller OML app to capture information and transmit to OMLserver Use LabWiki to observe FlowStats live

Sponsored by the National Science Foundation 33 GEC20 FlowStats - Exercise Can you modify this script to run UDP Iperf instead? What kind of Throughput graphs do you get? Can you change the Graph Definition to show aggregate number of flows per node? [Solution will be posted to the tutorial link by the end of this Tutorial] /Part2a/Execute

Sponsored by the National Science Foundation 34 GEC20 Want to Know More?