BIC 10503: COMPUTER ARCHITECTURE Chapter 2 Bus System (Part 2)

2.4 Interconnection Structure The collection of paths connecting various components is called Interconnection Structure

Interconnection Structure depends on the exchange of information in each computer modules. Types of exchange are indicated by input and output for each Computer Modules.

Types of exchanges/transfers: Memory to processor Processor reads an instruction or a unit of data from memory Processor to memory Processor writes a unit of data to memory I/O to processor Processor reads data from an I/O device via an I/O module Processor to I/O Processor sends data to the I/O device I/O to or from memory An I/O module is allowed to exchange data directly with memory without going through the processor using direct memory access

2.5 Bus Interconnection

Typically consists of multiple communication lines Bus Interconnection A communication pathway connecting two or more devices Key characteristic is that it is a shared transmission medium Signals transmitted by any one device are available for reception by all other devices attached to the bus If two devices transmit during the same time period their signals will overlap and become garbled Computer systems contain a number of different buses that provide pathways between components at various levels of the computer system hierarchy Typically consists of multiple communication lines Each line is capable of transmitting signals representing binary 1 and binary 0 The most common computer interconnection structures are based on the use of one or more system buses System bus A bus that connects major computer components (processor, memory, I/O)

Data Bus Data lines that provide a path for moving data among system modules May consist of 32, 64, 128, or more separate lines The number of lines is referred to as the width of the data bus The number of lines determines how many bits can be transferred at a time The width of the data bus is a key factor in determining overall system performance

Control Bus Address Bus Used to designate the source or destination of the data on the data bus If the processor wishes to read a word of data from memory it puts the address of the desired word on the address lines Width determines the maximum possible memory capacity of the system Also used to address I/O ports The higher order bits are used to select a particular module on the bus and the lower order bits select a memory location or I/O port within the module Used to control the access and the use of the data and address lines Because the data and address lines are shared by all components there must be a means of controlling their use Control signals transmit both command and timing information among system modules Timing signals indicate the validity of data and address information Command signals specify operations to be performed

Bus Interconnection Scheme

How do Buses look like? Parallel lines on circuit boards Ribbon cables Sets of wires Note: An on-chip bus connects the processor and internal cache memory. xxxx

Physical Realization of Bus Architecture This arrangement is convenient because (1) component on board can be easily removed and replaced (2) expand by adding more boards (Physically, the system bus is actually a number of parallel electrical conductors.)

Many devices on one bus leads to: Multiple Buses Single Bus Problems Many devices on one bus leads to: Propagation delays (more devices attached to a bus, the longer the path, the higher the propagation delay) Long data paths mean that co-ordination of bus use (to pass control from one device to another frequently) can adversely affect performance If aggregate data transfer approaches bus capacity (data transfer nearly reach maximum data rate that the bus can carry) Most systems use multiple buses to overcome these problems xxxx

Traditional Bus Architecture The cache memory is directly connected to the local bus in order to prevent the processor from accessing the slower main memory too frequently. In this way, I/O transfers to and from the main memory across the system bus do not interfere with the processor’s activity. This traditional bus architecture is reasonably efficient (e.g. for 10Mps Ethernet network card) but begin to break down due to growing demands of I/O devices. Expansion bus xxxx

High Performance Architecture Sometimes called as mezzanine architecture. This high speed bus can support high-speed LANs such as 100Mps Fast Ethernet network card. This bus is designed to support high-capacity I/O devices. Cache/ bridge High-speed bus xxxx

(cont) High Performance Architecture Lower-speed devices are supported through the expansion bus, linked by expansion bus interface between the expansion bus and the high-speed bus.

Elements of Bus Design

Element of Bus Design 1: Bus Types Dedicated (Physical Dedication) Permanently assigned to one function OR to a physical subset of computer components. e.g. Separate data & address lines (function dedication) which is common on many buses. e.g. Use an I/O bus to interconnect all I/O modules. This bus is then connected to the main bus through some sort of I/O adapter module. Advantage Higher throughput because there is less bus contention. Disadvantage Use more space (NOTE: Contention=the act of contending (competing)) xxxx

(cont.) Element of Bus Design 1: Bus Types Multiplexed Shared lines Use address valid control line or data valid control line e.g. when an address is transferred, the address valid control line is activated (one of the line is used as a signal or activation of an address transfer). Advantage Fewer lines (less bus lines), thus use less space and less cost. Disadvantage More complex control (complex circuitry) Reduce in performance (because some events cannot be done in parallel) xxxx

Element of Bus Design 2: Bus Arbitration Centralised A single hardware device, referred to as bus controller or arbiter, is responsible for allocating time on the bus. The device may be a separate module or part of the processor. Distributed Designate the processor or an I/O module as master. Both contain access control logic and act together to share the bus. Only one module may control bus at one time The master may then initiate a data transfer (e.g. read or write) with some other device. xxxx

Element of Bus Design 3: Timing Timing refers to the way in which events are coordinated on the bus. Buses either use Synchronous or Asynchronous timing Synchronous Timing Events determined by clock signals A single 1-0 transmission is a clock cycle or bus cycle Defines as Time slot Asynchronous Timing Transmission are not based on time An event occur in the bus depending on the occurrences of a previous event. Time slot Synchronous Timing Diagram xxxx

Synchronous Timing In this example, the processor places a memory address on the address line during the first clock cycle and may assert various status lines.

Asynchronous Timing – Read Diagram xxxx

Asynchronous Timing – Write Diagram

Element of Bus Design 4: Bus Width The wider the data bus, the greater the number of bits transferred at one time. The wider the address bus, the greater the range of locations that can be referenced.

Element of Bus Design 5: Data Transfer Type They are various data transfer types Data transfer for Read operation Write operation Read-modify-write operation Read-after-write operation Block data transfer

Peripheral Component Interconnect (PCI) Intel began work on PCI in 1990 for its Pentium-based systems. A popular high bandwidth, processor independent bus that can function as a mezzanine or peripheral bus Delivers good system performance for high speed I/O subsystems PCI Special Interest Group (SIG) Created to develop further and maintain the compatibility of the PCI specifications

Example PCI Configuration

2.6 Point-to-Point Interconnect

Point-to-Point Interconnect A conventional shared bus contribute to the difficulties of increasing bus data rate and reducing bus latency to keep up with the processors At higher and higher data rates it becomes increasingly difficult to perform the synchronization and arbitration functions in a timely fashion Point-to-Point Interconnection  Has lower latency, higher data rate, and better scalability

Quick Path Interconnect QPI Quick Path Interconnect An implementation of Point-to-Point Interconnect Characteristics of QPI: Multiple direct connections Direct pairwise connections to other components eliminating the need for arbitration found in shared transmission systems Layered protocol architecture These processor level interconnects use a layered protocol architecture rather than the simple use of control signals found in shared bus arrangements Packetized data transfer Data are sent as a sequence of packets each of which includes control headers and error control codes

Multicore Configuration Using QPI

QPI Layers

QPI Link Layer Flow control function Needed to ensure that a sending QPI entity does not overwhelm a receiving QPI entity by sending data faster than the receiver can process the data and clear buffers for more incoming data Performs two key functions: flow control and error control Operate on the level of the flit (flow control unit) Each flit consists of a 72-bit message payload and an 8-bit error control code called a cyclic redundancy check (CRC) Error control function Detects and recovers from bit errors, and so isolates higher layers from experiencing bit errors

QPI Routing and Protocol Layers Routing Layer Protocol Layer Used to determine the course that a packet will traverse across the available system interconnects Defined by firmware and describe the possible paths that a packet can follow Packet is defined as the unit of transfer One key function performed at this level is a cache coherency protocol which deals with making sure that main memory values held in multiple caches are consistent A typical data packet payload is a block of data being sent to or from a cache

Physical Interface of the Intel QPI Interconnect

QPI Multilane Distribution

PCI Express (PCIe) Point-to-point interconnect scheme intended to replace bus-based schemes such as PCI Key requirement is high capacity to support the needs of higher data rate I/O devices, such as Gigabit Ethernet Another requirement deals with the need to support time dependent data streams

PCIe Configuration

PCIe Protocol Layers As with QPI, PCIe interactions are defined using a protocol architecture. Physical: Actual wires carrying the signals, in the form of 0 and 1 during transmission and reception. Data link: Is responsible for reliable transmission and flow control. Data packets generated and consumed by the Data Link Layer (DLL) are called Data Link Layer Packets (DLLPs). Transaction: Generates and consumes data packets used to implement load/store data Data packets generated and consumed by the Transaction Layer (TL) are called Transaction Layer Packets (TLPs).

(cont) PCIe Protocol Layers

PCIe Multilane Distribution

Summary Point-to-point interconnect Computer components QPI physical layer QPI link layer QPI routing layer QPI protocol layer PCI express PCI physical and logical architecture PCIe physical layer PCIe transaction layer PCIe data link layer Computer components Computer function Instruction fetch and execute Interrupts I/O function Interconnection structures Bus interconnection Bus structure Multiple bus hierarchies Elements of bus design Chapter 3 summary.