1. Washington WASHINGTON UNIVERSITY IN ST LOUIS jdd@arl.wustl.edu SPC II Architecture

2. 2 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM SPC Architecture TI Sys. FPGA 64 MB Pentium Cache North Bridge APIC SPC Link 16 bit PCI BUS IPPOPP FPX SWITCH

3. 3 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM Proposed SPC-II Architecture TI IPPOPP SWITCH Link 128/256 MB Pentium Cache North Bridge APIC SPC-II 16/32 bit 32 bit PCI BUS South Bridge ISA Bus ISA Devices Super-IO BIOS FPGA 16 bit FPX

4. 4 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM IP Traffic in from Link TI APIC Link 16/32 bit 32 bit FPGA 16 bit FPX Normal IP Traffic Active IP Traffic BW fpx BW link

5. 5 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM IP Traffic out to Link TI APIC Link 16/32 bit 32 bit FPGA 16 bit FPX Normal IP Traffic Active IP Traffic BW fpx BW link

6. 6 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM Total Traffic BW SPC to/from FPX BW fpx = (1-p)X + 3pX Normal IP Traffic TI APIC 16/32 bit 32 bit FPGA 16 bit FPX Active IP Traffic BW fpx BW link BW on Link X <= BW link BW active = pX = Active Traffic (1-p)X = Normal IP Traffic TI APIC 16/32 bit 32 bit FPGA 16 bit FPX Normal IP Traffic Active IP Traffic BW fpx BW link

7. 7 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM Total Traffic BW on Link X <= BW link (1-p)X = Normal IP Traffic pX = Active IP Traffic BW SPC to/from FPX BW fpx = (1-p)X + 3pX If p = 0.1 BW link =1 Gb/s Then BW fpx <=.9(1Gb/s) +.3(1Gb/s) BW fpx <= 1.2Gb/s pBW active BW fpx.1100Mb/s1.2.2200Mb/s1.4.3300Mb/s1.6.4400Mb/s1.8.5500Mb/s2.0

8. 8 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM MSR VCI Allocation SPC Control and Download Data –IP –Active DQ FPX Control Debug Command

9. 9 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM MSR without FPX 200Local … Delivery 207 to CP 40 To Egress … SPC/FPX for 47 NH Delivery 61 DQ 64 Debug Msgs 62 Command Msgs 33 APIC Control Resp 142+(3*N) Download Response Data Switch Side Line Card Side Next Router or CP IP Route Adds Shim From 50 Previous or Hop 52 or … CP 59 From 200 … CP 2xx Generated by SPC WUGS Control 3232 WUGS Control

10. 10 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM MSR without FPX 62 Command Msgs 61 DQ 0x321 APIC Control 142+(3*N) Download Switch Side Line Card Side Next Router or CP 40 To Egress … SPC/FPX for 47 NH Delivery 60 DQ Copy for Monitoring DQ Summary DQ Cells or Summary 60 To CP Shim Processing To 50 Next or Hop 52 or … CP 59 200Local … Delivery 2xx to CP To 200 … CP 2xx Consumed by SPC WUGS Control 3232 WUGS Control

11. 11 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM MSR with FPX 200Local … Delivery 2xx to/from CP 50 To or FPX 52 for … IP 59 Routing 64 Debug Msgs 62 Command Msgs 33 APIC Control Resp 142+(3*N) Download Response Data Switch Side Line Card Side Next Router or CP From 50 Previous or Hop 52 or … CP 59 From 200 … CP 2xx 51 Active Pkts back to FPX Active Processing Generated by SPC WUGS Control 3232 WUGS Control With FPX nothing from Link side gets sent to SPC for processing

12. 12 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM MSR with FPX 62 Command Msgs 0x321 APIC Control 142+(3*N) Download Switch Side Line Card Side Next Router or CP 60 DQ Copy for Monitoring DQ Summary DQ Cells or Summary 60 To CP 51 Active Pkts From FPX 200Local … Delivery 2xx to CP 50 From or FPX 52 after … DQ 59 Processing To 50 Next or Hop 52 or … CP 59 To 200 … CP 2xx Active Processing Consumed by SPC WUGS Control 3232 WUGS Control With FPX, this possible DQ Summary is the only thing that goes out link side from SPC

13. 13 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM MSR with FPX Switch Side Line Card Side Next Router or CP 2 nd FPX RAD 116-123 2 nd FPX NID 108-115 1st FPX RAD 100-107 1st FPX NID 76-83

14. 14 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM SPC II FPGA with Two Ports SPC-II LC Switch PCI Bus Port Port 1Port 0 SPC-II FPGA APIC transit

15. 15 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM SPC II FPGA with Two Ports SPC-II FPXLCSwitch PCI Bus Port Port 1Port 0 SPC-II FPGA APIC transit

16. 16 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM SPC II FPGA with Two Ports SPC Only: –Link Side  Port 0 –Switch Side  Port 1 With FPX: –In from Link Side passes thru to Switch Side –Nothing goes out Port 0 to Link Side except for pesky DQ summary for monitoring which may or may not be done in SPC – In/Out from/to Switch Side small number of VCIs need to be handled

17. 17 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM SPC II FPGA with Two APIC Ports SPC-II CLOCK DOMAINS FPXLCSwitch APIC PCI Bus Port Port 1Port 0 SPC-II FPGA VPI[0]=1 VPI[0]=0 64<=VCI<=127???

18. 18 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM Reset of SPC II FPGA SPC-II CLOCK DOMAINS FPXLCSwitch APIC PCI Bus Port Port 1Port 0 SPC-II FPGA VPI[0]=1 VPI[0]=0 64<=VCI<=127??? VPI[0]=1 VCI = 38 Reset

19. 19 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM SPC II Issues Do we need to be able to receive APIC Control Cells from both the Link and Switch side? How do we operate an SPC II on test fixture in the lab? We have access to only the link side on the test fixture. How many APIC ID pins do we need? Reset: –If the FPGA generates reset based on a control cell what VPI/VCI does it arrive on? –Does the FPGA need to look at the payload or can we just say that anything arriving on the VPI/VCI is a reset cell?

20. 20 Washington WASHINGTON UNIVERSITY IN ST LOUIS John DeHart- 2/17/2016 5:02 AM SPC II FPGA with One APIC Ports SPC-II CLOCK DOMAINS FPXLCSwitch APIC PCI Bus Port Port 0 SPC-II FPGA