המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory הטכניון - מכון טכנולוגי לישראל הפקולטה להנדסת חשמל Technion - Israel institute of technology department of Electrical Engineering Full sniffer system for PCIe Preliminary Design Review Performed by: Omer Blecher, Roy Fridman Instructor: Boaz Mizrachi

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Agenda Main Goal Motivation Multi level level block diagram (sub level goals and risk assessment) Data flow Learning stages Dependencies Schedule

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Main Goal Providing a fully operational Sniffer who is able to connect on a PCIe bus,stream a PCIe packet to a analyzer and perform a complete packet analysis of the signals in an analysis and control PC. General purpose of project utilizing/modifying existing components of the system and creating missing components for full system integration).

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Why creating a sniffer: cost effective : experiment System available today works in sterile environment and doesn't “feel” real PCIe traffic. Can be modified and contain features with educational values The need for an integration project:company Price of analyzer +/ $ +/ $ +/ $ utilizes already spent/available lab resources Verify and modify existing tools

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory System overview PCI-e MB PCI-e Card Sniffing system x16 x1 Sniffing system x16 x1 x16 x1 PC

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Block Diagram ? ? ! ! ? ? ! ! ? ? ? ? ? ? Vertex II Pro RS232 PCIe Sniffer board ! ! PCIe PCI EXPRESS x16 link PCI-e Card PCI-e MB ? ? ? ?

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratorytasks Risk management Board testing and debugging: 1. Initial test of board components (“sanity test”). 2. Control path. 3. Data path - * loop back * x2 loop back * x2 loop back 4. Flow trough. 5. Multi cast. Supply voltage problem Distortion of PCI signal: Weak links are in the Data path test (loop back and x2 loop back). Learning PIC interface with EEPROM for future use (Low level drivers). Interface between PIC s/w and analysis and control PC s/w. Sniffer board back

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratorytasks Risk management Getting familiarized with the core concept and HDL design The Analyzer Core was not yet tested and only went trough simulation. Control VHDL programming with HDL designer The packet analyzer and exerciser was originated for experiment use and not “real- time“ sniffing Control power PC controller programming Modifying and merging designs according to analysis demeaned and selecting the most suitable evaluation board Analyzer core back

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratorytasks Risk management Gathering familiar analyzer and sniffer features in order to collect ideas and leads for GUI structure and content PC s/w uses all of the analysis and control PC resources Deciding on a programming language and leaning it Accessing the PIC s/w for the sniffer board from analysis and control PC s/w Modifying design according to analysis demeaned Making sure that storing data mechanism works Analysis and control PC back

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Data flow Vertex II Pro RS232 Sniffer board PCI EXPRESS x16 link PCIe PCI-e Card PCI-e MB Analysis and control PC PCIe start

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Dependencies So far the two other groups working on the project are in the following stage: Sniffer board (Roee Mesinger): 1.Current stage - starting layout. 2.Expected result - a board after assembly on 02/05. 3.Date of transfer – 04/06 (after debugging). Analyzer core (Danny Volkind and Amir Shmuel): 1. Current stage - finishing simulation in 3 weeks and ready to start building transmitter. 2. Expected result - basic function tested analyzer on 05/06 3. Date of transfer (if we will fully use the core) - 05/06

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Learning stages (beginning) PCIe protocol (packet analysis) I.P PCIe physical data transfer (on the connections between the system blocks. I.P Predecessor work: Analyzer core,TGA,sniffer board. I.P investigating known PCI and PCIe analyzers a and sniffers and creating a feature list for feature use. I.P

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory schedule Schedule also depends on other groups. Minimum time spent on project on a weekly base(toghter) – 2 full work days and 6 hour. 08/01/06 - Presenting final stage of system characterization 04/02/ Full report on the design/algorithm of analysis and control PC s/w. 2. Full report on the h/w design/algorithm mostly analyzer related) 3. Start of Debugging stages of the sniffer board with Roee Mesinger. 01/04/06 – Presenting Part A of the integration project: 1. Full control of all s/w and h/w design and test tools 2. Partly ready chosen architecture of analyzer and analysis and control PC s/w.

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Theoretical glossary(1/5) Device ADevice B The PCIe link is built around a bidirectional, serial (1-bit), point-to-point connection known as a "lane". At the electrical level, each lane utilizes two unidirectional low voltage differential signaling (LVDS) pairs at 2.5 gigabaud. Transmit and receive are separate diff-pairs, for a total of 4 data wires per lane. clock packet Selectable width packet TX+ TX - RX+ RX - RX+ RX - TX+ TX -

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Theoretical glossary(2/5) The basic data transfer method on the PCIe lines consists of a ”split transaction” protocol which is usually a “request” transaction a “completion” transaction. A basic packet is built in the following way: Transaction Layer Data Header Data Link Layer CRC Sequence Number Physical Layer Frame

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Theoretical glossary(3/5) There are 4 PCIe transaction types: Memory transaction. I/O transaction. Message transaction. Configuration transaction. I/O Write Request I/O Read Request I/O Write Completion Interrupt signaling Error signaling Power management I/O Read Completion Memory Write Request Memory Read Request Memory Write Completion Memory Read Completion Configuration Write Request Configuration Read Request Configuration Write Completion Configuration Read Completion

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Theoretical glossary(4/5) Electrical Physical Layer - High Speed Electrical Signaling Inbound clock significantly reduces EMI. AC coupled link by use of serial capacitors C TX and C RX Positioned closely to the transmitter of each lanes differential pair allowing the elimination of DC common mode voltage sharing between the two devices - separate transmitter DC common voltage. De-Emphasis prevents Inter-symbol interference as shown in the next example:

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Theoretical glossary(5/5) Pre-Emphasis example: capacitive effects on the link transmission are significant in 2.5Gbit/s and when changing polarity after being in a constant differential voltage, the line doesn’t easily loss the charge he collected (C pad +C interconnect +C tx ) and change the voltages. This is the inter-symbol interference The pre-emphasis allows overcoming this problem

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Analyzer Core back

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Packet Analyzer and Exerciser back

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory Analyzer core – block diagram Analyzer core (still in simulation) Currently designed for Memec evaluation board model FF672 Packet Analyzer and Exerciser - TGA (ready for use) Packet Analyzer and Exerciser - TGA (ready for use) Currently designed for Memec evaluation board model ? back

המעבדה למערכות ספרתיות מהירות High speed digital systems laboratory 5Vdc Sniffer board – block diagram M x34 PIC uController 18F MHz Dip Switch RESET Power supply 3.3Vdc 2.5Vdc 1.2Vdc Dip Switch Prog. Connector 4MHz Debug Leds UART RS232 TCVR RS232 EEPROM 24LC16B I²C Serial port interface (SPI) 64 ( 16 lanes) D type connector SMA connectors 4( 1 lane) 64 ( 16 lanes) back