Washington WASHINGTON UNIVERSITY IN ST LOUIS The New MSR Plugin Download Environment

2 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 First: The Old Way Compile plugin code on NetBSD Copy plugin directory onto File System destined for SPC Boot SPC Log in to SPC –requires user space and serial line or telnet access Use modload(1) on SPC to load plugin –links plugin against kernel in /netbsd on SPC Use modstat(1) on SPC to see list of loaded plugins

3 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 The New Way Compile plugin code on NetBSD Verify plugin has no undefined symbols –There is no link against running kernel anymore –wu_arl/msr/usr/src/sys/msr/msr_lkm.h provides macros to needed kernel functions and structures We can add more as we need them. Plugins do need to be recompiled if msr_lkm.* changes But if msr_lkm.* is stable, even if rest of kernel changes, plugin does not need to be recompiled. Use pluginDownload utility from CP (Linux) to download plugin to SPC –Only runs on Linux right now… Use instantiatePlugin utility from CP to config plugin, filters, etc Use pluginStat utility from CP to view list of loaded plugins

4 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 Exposed Functions Kernel functions –assert() –MALLOC() –FREE() Clock Functions –cpu_clock_1usec() –cpu_clock_10usec() –cpu_clock_1msec() LKM Functions –lkm_nofunc() –lkmdispatch() –lkmexists()

5 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 Exposed Functions (continued) MSR Plugin Control Unit –pcu_deregister_class() –pcu_free_all_instances() –pcu_register_class() –pcu_list_all_classes()

6 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 pluginDownload Uses gnu binary utilities cross-linker –Runs on Linux, builds a NetBSD binary –Used to: prelink to get size information final link to relocate Uses new AAL5 control data channels to download plugin directly into kernel memory. –uses sendcmd API to: allocate kernel memory for plugin set up AAL5 data channel call plugin entry function to complete load operation Side Note: sendcmd API –should be expanded to include all of sendcmd, but right now we have about 15 “commands”. Source files: wu_arl/utilities/SPC/plugDownload

7 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 pluginDownload: steps it takes –pre-link object file so we can get sizes use a bogus kernel address to force a.out file to have valid sizes –open pre-linked plugin file to get size –allocate two chunks of kernel memory 1: plugin code 2: plugin symbols returns addresses of two chunks and slot in lkm table –re-link plugin to relocate –open two dchans into MSR kernel plugin code plugin symbols –download plugin code and symbols –finish plugin loading by calling its entry function sendcmd -c rp_pcu –s load causes kernel to call entry function –release dchans

8 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 USAGE: pluginDownload pluginDownload [-h] [-d] [-V ] [-D -D ] [-s] -p -e -h : get this usage message -d : turn on debug output -V : specify the VPI/VCI to use for download. : overrides default of port# -D : specify the MSR Control Data Channel ids to use. : First use of -D is for file channel : second use of -D is for symbol table channel : if they aren’t specified, they’ll be allocated -p : specify the port to be downloaded to -e : specify the symbol name that should be called : when module is loaded -s : include symbols in the plugin download

9 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 USAGE: instantiatePlugin instantiatePlugin -p port [-h] [-ge] [-f filterId] pluginId flags srcAddr srcMask srcPort dstAddr dstMask dstPort protocol -p port : MSR Port to query -g : bind to general match classifier -e : bind to exact match classifier -f filterId : use this filter table entry (gen match classifier only) -h : Get this usage message

10 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 Example ../../../utilities/bin/$MYOS/pluginDownload -p 4 -e wvplugin -s combined.o # Gen plugin src src src dst dst dst # Port Cfy Fid Id flags addr mask port addr mask port prot  instantiatePlugin -d -p 4 -g -f

11 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 Extra Supporting Notes

12 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 Data Transfer Vaddr Start of Virtual address block where data frames should go (32b) Length Length of block in bytes (32b) Next Addr Where next frame should go. Initially set to (32b) Seq # Next expected frame sequence number (first frame should be 1) (16b) Index(dchan#) DCHAN_MAX MSR Kernel Dchannel Table

13 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart - 10/16/2015 Data Transfer (continued) MSR Control Data Channel RATM AAL5 Frame Format (as it appears in the MSR Kernel) dchan#(8bits)seq# (16bits) framelength (16bits)pad (16bits) AAL5 Trailer (8 Bytes) AAL5 Frame Data (<= 1992 Bytes) Shim Reserved space (8 Bytes) 1992 B 2000 B 2016 B APIC Rx desc points here flags (8bits) flags: 1: Data 2: Ack 4: Nack