Washington WASHINGTON UNIVERSITY IN ST LOUIS MSR Tutorial Group Exercises

2 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise Goals Get you familiar with the tools we use for configuring, testing and analyzing the MSR Get you familiar with building and running an MSR Not worry about a lot of MSR internal details

3 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 General Info Each group will be using 3 PCs –Data Source ($DSRC) Linux –Monitor Switch Control Processor ($SWCP), NetBSD –MSR Control Processor ($MSRCP), Linux All utilities work on both Linux and NetBSD You must use NetBSD when building: –MSR Kernels and Plugins The exercises will start out by giving you very specific directions. Later exercises leave out some details that you should be able to fill in from what you learned in earlier exercises.

4 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 General Info (continued) MSRs are configured with serial cables to SPCs on Ports 4 and 5. See aliases spc4 and spc5 in the readme file and your login script. We are going to be configuring and testing a router based on the WUGS and SPCs from the hardware on up. There are a LOT of details involved. Your login script defines macros/variables to simplify the configuration process. In particular the hosts are defined by the variables: –MSRCP, SWCP and DSRC

5 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries Prior to you doing the exercises we have already defined a few simplifying macros and built the software utilities: –See $MSRTUT/README for a complete list of variables and aliases –setenv REPODIR ~ –setenv WUARL ${REPODIR}/wu_arl –setenv WUGS ${WUARL}/wugs –setenv WUTILS ${WUARL}/utilities/ –setenv WUTUT ${WUARL}/TUTORIAL –setenv MSR ${WUARL}/msr –setenv MSRTUT ${WUTUT}/msr –cd ${WUGS}/src –gmake OSTYPE={Linux | NetBSD} depend –${WUARL}/Make.script install

6 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 IP Configuration for Exercises vci = 50 MSR-# Switch CP Host Data Src Host P4 P1 Monitoring Switch P3P1P2 P5 P0P4P6 P7 P3 NH MSR (P3) P0 MSR CP IP Router (Switch + SPCs + CP) vci = vci =

7 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Group Host Assignments Group # Data Source ${DSRC} (Linux) Switch CP ${SWCP} (NetBSD) MSR CP ${MSRCP} (Linux) Port 4 SPC serial line Port 5 SPC serial line 1mouserdemand5demand4 demand5 tip spc1 demand4 cu -l ttyS0 -s tabbydeakdemand0 deak tip spc0 demand0 cu -l ttyS0 -s mottinmvc2nmvc1 deak tip spc4 demand7 cu -l ttyS0 -s abyssiniangussiedemand3 nmvc2 tip spc1 tomcat cu -l ttyS0 -s tomcatnmvc3demand8 demand5 tip spc0 demand7 cu -l ttyS1 -s 9600

8 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - Host ATM config All steps are also recorded in the ${MSRTUT}/cfgs/README file. You can simple open this file, then cut and paste the commands Logging on to the Tutorial hosts: –login: msr# password: Msr#Tut –root password, where needed: TutR00T (those are zero’s)

9 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - Host ATM config 1.Login to the Data Source host, open a local window (Linux) open local window - use the alias ‘local’ defined in login script alias local '/usr/X11R6/bin/xterm -j -sb -sl bw 3 -T "Local" &' go to config directory and become the root user cd ${MSRTUT};su Assuming atmarpd has not be started, /etc/init.d/atm start ps -agx | grep | egrep "atmsigd|ilmid“ kill Configure the APIC interface./apic.cfg port 4 exit root exit

10 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - Host ATM config 2.Open window on the monitoring Switch CP (NetBSD Host) use alias swcp, this is already defined in your login script alias swcp 'ssh -f $SWCP /usr/X11R6/bin/xterm -ls -j -sb -sl bw 3 -T "SWCP\($SWCP\)"' go to config directory and become the root user cd ${MSRTUT}/cfgs; su configure local APIC interface for the switch cp./apic.cfg port 1 swcp exit root user exit Start up GBNSC and configure Monitoring Switch ${WUGS}/bin/NetBSD/newGBNSC.init -P3551 -init init.GBNSC.switch1 config.GBNSC.switch1 & ${WUGS}/bin/NetBSD/Jammer 0.1 ${SWCP} 3551 group.js

11 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - Host ATM config 3.Open window on the MSR CP (Linux Host) use the alias ‘msrcp’ defined in your login script alias msrcp 'ssh -f $MSRCP /usr/X11R6/bin/xterm -j -sb -sl bw 3 -T "MSRCP\($MSRCP\)"‘ go to config directory and become the root user cd ${MSRTUT}/cfgs;su Assuming atmarpd has not be started, /etc/init.d/atm start ps -agx | grep | egrep "atmsigd|ilmid“ kill configure local APIC interface for the switch cp./apic.cfg port 0 exit root user exit

12 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - Building a kernel Go to your Switch CP host window (must be NetBSD) The following environment variables are defined in your login script SYS = ${MSR}/usr/src/sys CONF = ${SYS}/arch/i386/conf DKERN =${SYS}/arch/i386/compile/MSR_SPC PKERN = ${SYS}/arch/i386/compile/MSR_SPC_PERF Building a kernel (On NetBSD) –cp ${MSRTUT}/FS/SPC24MB.fs /usr/MSR –Debug kernel 1.cd $CONF; config MSR_SPC 2.cd $DKERN; make depend;make –Performance kernel 1.config MSR_SPC_PERF 2.cd $DKERN;make depend;make

13 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - Building a kernel Copy kernel symbols into the filesystem (still on SWCP - NetBSD) –cp ${MSRTUT}/FS/SPC24MB.fs /usr/MSR –su # become the root user vnconfig -t spc24MB -v -c /dev/vnd0d SPC24MB.fs mount /dev/vnd0d /mnt ${WUTILS}/bin/NetBSD/mknl ${DKERN}/netbsd /mnt/netbsd or ${WUTILS}/bin/NetBSD/mknl ${PKERN}/netbsd /mnt/netbsd umount /mnt vnconfig -u /dev/vnd0d –exit –cp /usr/MSR/SPC24MB.fs ${MSRTUT}/FS/SPC24MB.fs

14 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - SPC Kernel continued Go to the MSRCP window. Copy kernel and filesystem to the staging area –cd /usr/MSR –cp ${MSRTUT}/FS/SPC24MB.fs /usr/MSR –rm /usr/MSR/netbsd.MSR –cp ${DKERN}/netbsd /usr/MSR/netbsd.DEBUG or cp ${PKERN}/netbsd /usr/MSR/netbsd.PERF –ln -s /usr/MSR/netbsd.DEBUG /usr/MSR/netbsd.MSR or ln -s /usr/MSR/netbsd.PERF /usr/MSR/netbsd.MSR

15 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - MSR Configuration Go to the MSRCP window –Download kernels and configure switch cd ${MSR}/cp/configuration./Linux/MSR_Config -f config.MSR – set port (SPC) default parameters cd ${MSRTUT}/cfgs./defconfig.cmd –start up GBNSC for use by the GUI monitoring tool ${WUGS}/bin/Linux/newGBNSC -P3551 config.GBNSC.MSR & Start up the GUI: go to the Data Source Host’s local window –/pkg/jdk1.2.2/bin/java -jar ${MSR}/apps/javaGUI/pubgui.jar –then load the msrgui.txt configuration file and arrange windows

16 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Preliminaries - Traffic Generation We will use sendpkts for all exercises in the MSR tutorial sendpkts - uses default socket interface and requires configuration of IP and ATM VCs on data sources –${WUTILS}/bin/Linux/sendpkts -S -n 10 -a 1 –l AAL5Generator - uses APIC User mode library and cannot coexist on the same VCIs as sendpkt, –undo atm: su; sh ${WUGRP}/cfgs/IP/DataSource.atm.undo;exit –${WUTLS}/AAL5Generator -pacer 8 \ -src dst \ -frate svci 50 -seconds 20 -packet 20 –Use sendCommand utility to change AAL5Generator’s output Sending commands to the ports using sendcmd and monitoring debug messages with monmsgs. –${MSR}/bin/Linux/monmsgs & –${MSR}/bin/Linux/sendcmd -p 4 -c policy -s set_dflags -d 0x30 -t 10000

17 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1 - Part 1 Update the Tutorial Directory 1.cd $MSRTUT 2.cvs update -A -d Default routes using the built in Simple route module and table 3.copy $MSRTUT/cfgs/hosts to /etc/hosts on both the SWCP and DSRC hosts 4.in the MSR CP window, Download a DEBUG kernel and verify 5.In the Data Source window generate IP traffic from Data Source ( ) to port 5 of the MSR and verify with the GUI $WUTILS/bin/Linux/sendpkts -x p 100 -a 1 -S -s l

18 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1: Part 1 Bidirectional traffic using an echo server on SWCP 6.In the SWCP window, start up a IP packet echo server $WUTILS/bin/NetBSD/serv -p e 1 This will echo all packets received back to he sendpkt command 7.In the DSRC window, send packets to the SWCP host ( ) $WUTILS/bin/Linux/sendpkts -x p 100 -a 1 -S -s l this command sends single cell packets at a rate of 1000 per second. Verify with the GUI that packets are “flowing” in both directions.

19 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1, Part 1: Default Routes vci = 50 MSR-# Switch CP Host Data Src Host P4 P1 Monitoring Switch P3P1P2 P5 P0P4P6 P7 P3 NH MSR (P3) P0 MSR CP probe (A) probe (B) vci = vci = probe (C) IP Forwarding Simple, Default routes, at Port 4

20 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1, Bidirectional Traffic vci = 50 MSR-# Switch CP Host Data Src Host P4 P1 Monitoring Switch P3P1P2 P5 P0P4P6 P7 P3 NH MSR (P3) P0 MSR CP IP Forwarding using FIPL at Port probe (A) probe (B) vci = Destination Route = OutVIN =, Maps to Port 1, VCI = 50 probe (C) serv (echo) sendpkts

21 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1, Part 2 Enable the FIPL module and define the routing tables –In the MSRCP window turn fipl on at port 4 ${MSR}/bin/Linux/sendcmd –p 4 –c policy –s set_fipl –t Install routes port 4, add route to port 5, subport 0 ${MSR}/bin/Linux/fip –s –p 4 Enter command (h for help): a / –Verify, in the DSRC Window execute the command $WUTILS/bin/Linux/sendpkts -x p 100 -a 1 -S -s l Verify traffic follows correct route with the GUI

22 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1, Part 3 Add routes from port 4 to port 1, subports 0 and 1 –in the MSRCP window add routes to existing fip session Enter command (h for help): a / Enter command (h for help): a /24 65 quite fip Enter command (h for help): quite Define the return routes from port 1 to port 4 – in the MSRCP window To simplify the process, the script fipl.cmd enables fipl and defines default routes to all virtual interfaces. $MSRTUT/cfgs/fipl.cmd port 1 or to enable fipl and define routes on all ports $MSRTUT/cfgs/fipl.cmd

23 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1, Part 3 Send packets from the data source (DSRC) to the Switch CP host (SWCP) at its alias IP address on subnet –Open a second local window on the data source host –in one window send packets to $WUTILS/bin/Linux/sendpkts -x p 100 -a 1 -S -s l –in the other window send packets to $WUTILS/bin/Linux/sendpkts -x p 100 -a 1 -S -s l –Verify bidirectional traffic using the GUI - the echo server should still be running on the switch CP host.

24 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 1, Part 3: SubPorts vci = 50 MSR-# Switch CP Host Data Src Host P4 P1 Monitoring Switch P3P1P2 P5 P0P4P6 P7 P3 NH MSR (P3) P0 MSR CP probe (A) probe (C) probe (B) vci = 51 vci = Destination Route = OutVIN =, Maps to Port 1, VCI = 51 Returning traffic not shown

25 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 2: Fair Queuing In SWCP window, stop the echo server (do a ^C in the window) –Restart the server but without the echo option $WUTILS/bin/NetBSD/serv -p e 0 Set interface and internal rates (internal rate includes a speed advantage) –in the MSRCP window $MSR/bin/Linux/sendcmd -p 1 -c policy -s set_trate -d d 0 -d 150 -t w info 0) (0xc350)# G = Max rate for APIC interface 1) (0x2710)# T = Total Rate = Link Rate + Internal Rate 2) 4000 (0xfa0)# L = Link Rate (aka External rate) 3) 150 (0x96)# S’ = Speedup factor (percent) 4) 6000 (0x1770)# Internal Rate (port to port rate) –All rates are in Kbps.

26 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 2: Fair Queuing Default behavior is effectively a FCFS, tail drop packet scheduler. In the two Data source windows, send traffic and observe packet scheduling behavior in the GUI –turn on source 1, approx 3.4 Mbps $WUTILS/bin/Linux/sendpkts -c -x p 100 -a 1 -S -s l –in other window turn on source 2, approx 2.5Mbps $WUTILS/bin/Linux/sendpkts -c -x p 100 -a 1 -S -s l With monitoring the GUI, turn on Queue State DRR –In the MSRCP window, send command to enable qsdrr on port 1 $MSR/bin/Linux/sendcmd -p 1 -c policy -s set_ps -d 2 The two flows not fairly share available BW, go boack to the FCFS scheduler (in the MSRCP window) –$MSR/bin/Linux/sendcmd -p 1 -c policy -s set_ps -d 0

27 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Compiling and downloading a plugin –in the SWCP window (Need NetBSD), build the object file (combined.o) cd ${MSR}/rp/plugins/msr_template make –Then in the MSR CP window, download plugin to running SPC kernel (msr_template is hte entry point name) cd ${MSR}/rp/plugins/msr_template setenv WU_ARL_ROOT $WUARL $WUTILS/bin/Linux/pluginDownload -p 5 -e msr_template -s combined.o Plugin successfully loaded with module ID: 0 –The downloader prints out the plugin id number. You will need this in later commands.

28 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Next, turn on debug messages for the plugin, classifier and pcu modules –In the MSR CP window $MSR/bin/Linux/sendcmd -p 5 -c set_debug -l verbose -m plugin -m ctl -m pcu -m classify ## Create an instance of the plugin Turn on the general match classifier –In the MSR CP window $MSR/bin/Linux/sendcmd -p 5 -c policy -s set_gen -d 1 Create one instance of the plugin –In the MSR CP window $MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s create -i 100 -w info this creates instance number 0 of the plugin (see debug messages or the return value with -w info is used with the sendcmd command)

29 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins For a plugin to be called, it must be bound to either a general or exact match filter. In this exercise we use the general match filters - the exact match interface is still “sensitive” –Now create a general match filter that the plugin will be bound to. –addfltr - Add fltr: gid, fid, flags, src/msk, sp, dst/msk, dp, pr –gid = 0 for general match and 1 for exact match –use flags = 0 and set filter id to 0 for the first filter –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s addfltr -g 0 -f 0 -F 0 -n 0 -n 0 -d 0 -n n d 0 -d 0 -f is the filter id for use when we bind the plugin This will match any packets with a destination network prefix equal to /24.

30 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Now that we have a plugin instance and a general match filter is installed, we need to bind the plugin instance to the filter –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s bind -i 0 -g 0 -f 0 This binds plugin instance ID 0 to general match filter ID 0. Send a message to the bound plugin. The plugin will both printout and return the number of packets it has seen –$MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 0 Send packets that will match the filter. –In one of the the Data Source windows, send 10 packets $WUTILS/bin/Linux/sendpkts -n 10 -a 1 -S -s l Then send message to plugin and read the number of packets –MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 0 -w info

31 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Get a list of loaded classes, instances and filters - prints to SPC window –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s clist -t w info –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s ilist -t w info –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s flist -g 0 -w info Unbind the instance –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 0 -g 0 -f 0 Free the instance –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s free -i 0 Remove general match filter –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 0 -g 0 Unload a plugin (ignore error message) –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unload -d 0 Check on plugin status –$WUTILS/bin/Linux/pluginStat -p 5 0

32 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Download the plugin a second time and and create three instances –cd $MSR/rp/plugins/msr_template/ –$WUTILS/bin/Linux/pluginDownload -p 5 -e msr_template -s combined. o –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s create -i 100 -w info Create three filters –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s addfltr -g 0 -f 1 -F 0 -n 0 -n 0 -d 0 -n n d 0 -d 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s addfltr -g 0 -f 1 -F 0 -n 0 -n 0 -d 0 -n n d 0 -d 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s addfltr -g 0 -f 2 -F 0 -n 0 -n 0 -d 0 -n n d 0 -d 0

33 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Bind instances –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s bind -i 0 -g 0 -f 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s bind -i 1 -g 0 -f 1 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s bind -i 2 -g 0 -f 2 Now send packets so that each of the plugins will receive a different number of packets. –In the Date Source Window $WUTILS/bin/Linux/sendpkts -n 5 -a 1 -S -s d l $WUTILS/bin/Linux/sendpkts -n 8 -a 1 -S -s d l $WUTILS/bin/Linux/sendpkts -n 11 -a 1 -S -s d l

34 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Send messages to the different instances –$MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 1 –$MSR/bin/Linux/sendcmd -p 5 -c rp_inst -d 2 Print out information about loaded classes, instances and fliters: –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s clist -t w info –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s ilist -t w info –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s flist -g 0 Unbind instances –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 0 -g 0 -f 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 1 -g 0 -f 1 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unbind -i 2 -g 0 -f 2

35 Washington WASHINGTON UNIVERSITY IN ST LOUIS Fred Kuhns - 10/4/2015 Exercise 3: Plugins Free instances –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s free -i 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s free -i 1 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s free -i 2 Remove filters –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 0 -g 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 1 -g 0 –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s remfltr -f 2 -g 0 Unload class –$MSR/bin/Linux/sendcmd -p 5 -c rp_pcu -s unload -d 0