Running A First Example: The Web Bumper INF5060: Multimedia data communication using network processors Running A First Example: The Web Bumper 3/9 - 2004

Overview Packet header and encapsulation review How to use the card programming model booting starting and stopping Lab setup Web bumper

Packet Headers and Encapsulation

Ethernet 48 bit address configured to an interface on the NIC on the receiver 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source address | | Frame type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 48 bit address configured to an interface on the NIC on the sender describes content of ethernet frame, e.g., 0x0800 indicates an IP datagram

Internet Protocol version 4 (IPv4) indication of the abstract parameters of the quality of service desired – somehow treat high precedence traffic as more important – tradeoff between low-delay, high-reliability, and high-throughput – NOT used, bits now reused indicates the format of the internet header, i.e., version 4 length of the internet header in 32 bit words, and thus points to the beginning of the data (minimum value of 5) datagram length (octets) including header and data - allows the length of 65,535 octets first zero, fragments allowed and last fragment 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | | Identification |Flags| Fragment Offset | | Time to Live | Protocol | Header Checksum | | Source Address | | Destination Address | | Options | Padding | identifying value to aid assembly of fragments indicate where this fragment belongs in datagram disable a packet to circulate forever,decrease value by at least 1 in each node – discarded if 0 checksum on the header only – TCP, UDP over payload. Since some header fields change(TTL), this is recomputed and verified at each point indicates used transport layer protocol 32-bit address fields. May be configured differently from small to large networks options may extend the header – indicated by IHL. If the options do not end on a 32-bit boundary, the remaining fields are padded in the padding field (0’s)

UDP port to identify the sending application port to identify receiving application 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | | Length | Checksum | specifies the total length of the UDP datagram in octets contains a 1’s complement checksum over UDP packet and an IP pseudo header with source and destination address

TCP code bits: urgent, ack, push, reset, syn, fin sequence number for data in payload acknowledgement for data received port to identify the sending application port to identify receiving application 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | | Sequence Number | | Acknowledgment Number | | Header| |U|A|P|R|S|F| | | length| Reserved |R|C|S|S|Y|I| Window | | | |G|K|H|T|N|N| | | Checksum | Urgent Pointer | | Options | Padding | receiver’s buffer size for additional data header length in 32 bit units pointer to urgent data in segment contains a 1’s complement checksum over UDP packet and an IP pseudo header with source and destination address options may extend the header. If the options do not end on a 32-bit boundary, the remaining fields are padded in the padding field (0’s)

Encapsulation

Using IXP1200

Programming Model Active computing elements (ACEs): basic programming abstraction defined by Intel’s software developer kit (SDK), not part of hardware asynchronous unit of execution a typical system contains at least three in a pipeline: unidirectional queues between ACEs allow processors to run independently late bindings are used to bind ACE output at run-time (unbounded targets can be used to drop packets) ingress ACE process ACE egress ACE input ports output

Programming Model Packet flow illustration for IP forwarding: Stack ACE ingress ACE (core) IP ACE (core) egress ACE (core) StrongARM erroneous packet, not IP, … packet for this host microengine ingress ACE (microblock) IP ACE (microblock) egress ACE (microblock) input ports output ports packet is not for me, forward packet is an error free IP datagram

Programming Model Often many possible communication paths, but only one used per packet Division of work between ACEs running both on microengines and StrongARM crucial for high performance: microengines comprise fast data path – the common case StrongARM handles all exceptions

Programming Model Challenge: achieve optimal parallelism efficient resource utilization  each part of the pipeline should spend the same amount of time to process a packet, reduce idle time which microengine runs which ACE, many possible configurations, e.g., exactly one microblock ACE per engine a pipeline of microblocks for ACEs on one or more engines microblock groups: set of microblocks that run on a single engine specified in a configuration file

Programming Model Microblock group: Replicated microblock groups ingress ACE process ACE egress ACE input ports output microengine 1 microengine 2 ingress ACE process ACE input ports microengine 1 egress ACE output ports ingress ACE process ACE input ports microengine 3 microengine 2

Programming Model Microblock dispatch loop control packet flow among ACEs in a microblock group runs in an infinite loop, e.g., IP forwarder: initialize microblocks while (1) { get next packet from input port invoke ingress microblock if not IP packet send to ingress core else { invoke IP microblock if packet not is for this host send to egress microblock else send to IP core } } ingress ACE (microblock) IP ACE egress ACE (core) Stack ACE

Programming Model Exceptions (interrupts raised by microengines) refer to packets passed from microblock components (microengine) to a core component (StrongARM) the symbolic constant IX_EXCEPTION can be used components are identified using a tag an exception code informs the core component of the type core component must have an exception handler determining how to process the packet

Programming Model Crosscalls enable an ACE to invoke a function in another ACE similar to remote procedure calls (RPC) must be programmed into both caller and callee early binding, programmer determines type (not run-time) specified by declaring a function three types deferred – caller do not block ; asynchronous return notification oneway – caller do not block ; no return value twoway – caller blocks until callee returns value

File Access, Communication and Booting PCI Ethernet emulation physically, the card plugs into the host’s PCI bus host and board communicates sending Ethernet frames over the PCI The disk is mounted using NFS over PCI, e.g., /opt/ixasdk is available at both host and card The card itself has no persistent memory a RAM disk for the root file system is when booting the card, an image is loaded from the host bootixp performs these instructions and boots the card allow some time after booting Rebooting the card from the board (reboot) only clears/reinitialize the card  must run bootixp from host afterwards Running bootixp on a running card will crash the card boot (reinitialize) card through minicom, followed by a bootixp, or host reboot

Environment Requirements The Makefiles assume that the following environment variables are set: IXROOT = /opt/ixasdk CONFIG = arm_be … this is already set for the root user on the lab machines

Starting and Stopping Telnet to the card: telnet ixp To start the example: ixstart <config_file> our example config files use relative paths cd $IXROOT/bin/arm_be ; ./ixstart ixsys.config To stop the example: ixstop stops whatever ACE that is running cd $IXROOT/bin/arm_be ; ./ixstop

Lab Setup

Lab Setup … Reboot Error Internet ssh connection IXP lab switch switch Student lab switch power switch “reboot x”

Lab Setup – Power Switch IXP lab switch switch … power switch

Lab Setup - Addresses IXP lab switch … power switch

Lab Setup - Addresses … … … … IXP lab IXP lab switch power switch 129.240.67.111 192.168.67.111 switch switch 129.240.67.112 192.168.67.112 192.168.67.5 129.240.67.113 192.168.67.113 … … … 129.240.67.118 192.168.67.118 power switch 129.240.67.110

Lab Setup – Data Path … IXP lab IO switch switch hub memory hub CPU PCI ssh connection (129.240.67.112) switch IO hub switch hub interface memory hub local network (192.168.67.112) To 192.168.67.5 … system bus CPU IXP1200 RAM interface To 192.168.67.5 memory power switch web bumper (counting web packets and forwarding all packets from one interface to another)

Web Bumper

Lab Setup – Data Path … IXP lab IO switch switch hub memory hub CPU ssh connection (129.240.67.112) switch IO hub switch memory hub local network (192.168.67.112) … CPU IXP1200 To 192.168.67.5 memory power switch web bumper (counting web packets and forwarding all packets from one interface to another)

The Web Bumper the wwbump coreACE checks a packet forwarded by the microACE: count web packet - add 1 to counter send back to wwbump microACE provides access to counter via a crosscall server the wwbump microACE checks all packets if it is a web packet: if not, forward packet to egress microACE (and sent to the other port) if web packet: send to wwbump coreACE (StrongARM) forward packets from core to egress microACE (and sent to the other port) the wwbump microACE checks all packets if it is a web packet: if not, forward packet to egress microACE (and sent to the other port) if web packet: send to wwbump coreACE (StrongARM) forward packets from core to egress microACE (and sent to the other port) web bumper wwbump (core) IXP1200 StrongARM microengines input port ingress ACE (microblock) wwbump (microblock) egress ACE (microblock) output port web bumper (counting web packets and forwarding all packets from one interface to another) ingress processing encompasses all operations performed as packets arrive egress processing encompasses all operations applied as packets depart

The Web Bumper IXP1200 the crosscall client reads the web packet counter and prints the current count web bumper crosscall client wwbump (core) StrongARM microengines input port ingress ACE (microblock) wwbump (microblock) egress ACE (microblock) output port

Configuration File It is run at boot time and specifies ACEs to be loaded which side to run an ACE (ingress or egress) number and speed of ports see ixsys.config The interfaces that are to be started at system boot time: interface 0 10.1.0.1 10.1.0.255 255.255.255.0 00:01:02:03:04:05 1 interface 1 10.2.0.1 10.2.0.255 255.255.255.0 00:01:02:03:04:06 1 … Using workbench or not (we do not) mode 0

Configuration File The microcode and microaces that are to be started at system boot time file 0 ./SlowIngressWWBump.uof ingress microcode (type 0) associated slow ports (10/100 Mbps) microace ifaceInput ./ingressAce none 0 1 ifaceInput is a microace executable no config file running on ingress side type ingress microace wwbump ./wwbump none 0 0 runs on ingress side (first zero) has unknown type (second zero) …

Configuration File Bind configuration bind static ifaceInput/default wwbump binding for ingress (ifaceInput) is wwbump bind static wwbump/default ifaceOutput binding for wwbump output is ifaceOutput Shell commands to be run, e.g., add ARP entries in the cache

Identifying Web Packets These are the header fields you need for the web bumper: Ethernet type 0x800 IP type 6 TCP port 80

WWBump Dispatch Loop initialize dispatch loop macros do forever { if packet has arrived from StrongARM send to egress if packet has arrived from Ethernet port { invoke WWBump macro to process packet if it is a web packet (exception specified) send to StrongARM else send to egress } }

WWBump Macro allocate 6 SDRAM registers to hold header data get IXP input port  set IXP output port to the other read first 24 bytes into the 6 SDRAM registers (etype, IP hlen, IP type) if not frame type is IP (0x800), forward packet if not IP type is TCP (6), forward packet calculate port number address using IP header length read TCP destination port number if not TCP destination port is 80, forward packet set exception code set wwbump tag set IX_EXCEPTION return

Testing Log in on the assigned IXP machine (129.240.67.xxx) using ssh as root (twice, the example is easier using two different windows) Download code and place at a convenient place under $IXROOT/src, e.g.: cd $IXROOT/src/microaces/aces/ scp –r user@login.ifi.uio.no:/hom/paalh/INF5060code/wwbump bumptest Compile the code and make sure the targets are moved to $IXROOT/bin/arm_be microcode for the microengines: cd $IXROOT/src/microaces/aces/bumptest/ucbuild; make ; cp *.uof $IXROOT/bin/arm_be/. c-code for the StrongARM: cd .. ; make (wwbump and getcount are automatically put right directory) Copy configuration file: cp ixsys.config $IXROOT/bin/arm_be/ixsys.config-bumptest Telnet to the IXP card as root: telnet ixp Enter bin directory: cd $IXROOT/bin/arm_be/ (you may already be there!!) Stop any running aces: ./ixstop Start the bumper: ./ixstart ixsys.config-bumptest

Testing The bumper should now be running: In IXP window, check web packet count (should be 0): ./getcount In HOST window ping web server machine: ping 192.168.67.5 (packets should be forwarded through the card, but not counted, i.e.: ./getcount  0) telnet to web server machine using port 80: telnet 192.168.67.5 80 (packets should be forwarded through the card, and counted, i.e.: ./getcount  4 (!!??)) Experiment with the card: … start and stop the aces … reboot the card … remotely restart the machine using the power switch

Summary We’ll use remote access to the IXP lab machines Bumper shows the basic concepts Intel SDK and programming model microengines and StrongARM much code even for a small, trivial task Next: short demo (here) and testing (in the lab) Next week: extending the bumper example