Introduction to Programmable Logic Devices and FPGAs 11/6/2017 Introduction to Programmable Logic Devices and FPGAs Edward Freeman STFC Technology Department Detector & Electronics Division

Lecture Outline Introduction FPGA Field Programmable Gate Array 11/6/2017 Introduction Why Programmable Logic Devices and FPGAs FPGA Field Programmable Gate Array Architecture Design Flow Hardware Description Languages Design Tools ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Why Programmable Logic Devices and FPGAs 11/6/2017 Programmable Logic is a Key Underlying Technology for PP Experiments. First-Level and High-Level Triggering Data Transport (Networks) Computers interacting with Hardware (Networks) Silicon Trackers (Millions of Data Channels) Commercial Devices. Developments driven by Industry. Telecomms, Gaming, Aerospace, Automotive, Set-top boxes…. ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

CMS DAQ/Trigger Architectures 11/6/2017 CMS Fully custom PP ASICs Programmable Logic DIGITAL CPUs Commodity PCs ‘outline’ of coverage A full DAQ ayatem time-plan - What is an FPGA - And why are they important for HEP questions - any time “Telecoms Network” ~ 1 Tbps edward.freeman@stfc.ac.uk

Particle Physics Electronics 11/6/2017 Special Dedicated Logic Functions (not possible in CPUs) Ultra Fast Trigger Systems (Trigger Algorithms) Clock Accurate Timing Massively Parallel Data Processing (Silicon Trackers with Millions of Channels) Custom Designed Printed Circuit Boards PCBs. ‘outline’ of coverage What they look like time-plan - What is an FPGA - And why are they important for HEP questions - any time Commercial Programmable Logic Devices, FPGAs edward.freeman@stfc.ac.uk

Lecture Outline FPGA Field Programmable Gate Array Introduction 11/6/2017 Introduction Why Programmable Logic Devices and FPGAs FPGA Field Programmable Gate Array Architecture Design Flow Hardware Description Languages Design Tools ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Digital Logic < 40 nm ! $$$ Logic Gates MOORE’S LAW 11/6/2017 Logic Gates MOORE’S LAW ‘outline’ of coverage moore’s law time-plan - What is an FPGA - And why are they important for HEP questions - any time Transistor Switches < 40 nm ! $$$ edward.freeman@stfc.ac.uk

Digital Logic Digital Logic Function Product AND (&) Sum OR (|) 11/6/2017 Digital Logic Function Product AND (&) Sum OR (|) 3 Inputs SUM of PRODUCTS Black Box Truth Table (Look Up Table LUT) ‘outline’ of coverage 3 input logic func time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Digital Logic Black Box SUM of PRODUCTS Truth Table 11/6/2017 Black Box SUM of PRODUCTS Truth Table (Look Up Table LUT) Digital Logic Function 3 Inputs Product AND (&) Sum OR (|) ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Logic Blocks Logic Functions implemented in Look Up Table LUTs. 11/6/2017 Logic Functions implemented in Look Up Table LUTs. Flip-Flops. Registers. Clocked Storage elements. Multiplexers (select 1 of N inputs) ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time FPGA Fabric Logic Block edward.freeman@stfc.ac.uk

Look Up Tables LUTs 11/6/2017 LUT contains Memory Cells to implement small logic functions Each cell holds ‘0’ or ‘1’ . Programmed with outputs of Truth Table Inputs select content of one of the cells as output Configured by re-programmable SRAM memory cells 3 Inputs LUT -> 8 Memory Cells 3 – 6 Inputs ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time Static Random Access Memory SRAM cells Multiplexer MUX edward.freeman@stfc.ac.uk

Logic Blocks 11/6/2017 Larger Logic Functions built up by connecting many Logic Blocks together ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Logic Blocks 11/6/2017 Larger Logic Functions built up by connecting many Logic Blocks together Determined by SRAM cells SRAM cells ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Sequential Circuits 11/6/2017 Combinational Logic (Larger circuits difficult to predict) Synchronous Logic driven by a CLOCK Registers, Flip Flops (Memory) Intermediate New Output every clock edge Inputs Clock Rate determines speed Comb Logic Must meet Timing => Predictable circuits Register CLOCK ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time Shift Registers, Pipelines, Finite State Machines … EDGES edward.freeman@stfc.ac.uk

Clocked Logic Registers on outputs. CLOCKED storage elements. 11/6/2017 Registers on outputs. CLOCKED storage elements. Synchronous FPGA Logic Design, Pipelined Logic. FPGA Fabric Pulse from Global Clock (e.g. LHC BX frequency) ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time FPGA Fabric Special Routing for Clocks Clock from Outside world (eg LHC bunch frequency) edward.freeman@stfc.ac.uk

Field Programmable Gate Arrays FPGA 11/6/2017 Field Programmable Gate Array ‘Simple’ Programmable Logic Blocks Massive Fabric of Programmable Interconnects Standard CMOS Integrated Circuit fabrication process as for SRAM memory chips (Moore’s Law) “Hard blocks” for complex high speed functions Huge Density of Logic Block ‘Islands’ 1,000 … 100,000’s in a ‘Sea’ of Interconnects FPGA Architecture A field-programmable gate array (FPGA) is a large-scale integrated circuit that can be programmed after it is manufactured rather than being limited to a predetermined, unchangeable hardware function. The term "field-programmable" refers to the ability to change the operation of the device "in the field," while "gate array" is a somewhat dated reference to the basic internal architecture that makes this after-the-fact reprogramming possible. edward.freeman@stfc.ac.uk

Field Programmable Gate Arrays FPGA 11/6/2017 A field-programmable gate array (FPGA) is a large-scale integrated circuit that can be programmed after it is manufactured rather than being limited to a predetermined, unchangeable hardware function. The term "field-programmable" refers to the ability to change the operation of the device "in the field," while "gate array" is a somewhat dated reference to the basic internal architecture that makes this after-the-fact reprogramming possible. edward.freeman@stfc.ac.uk

Input Output I/O Getting data in and out 11/6/2017 Input Output I/O Getting data in and out Up to > 1,000 I/O “pins” (several 100 MHz) Special I/O SERIALISERS ~ 10 Gbps transfer rates Optical TRx edward.freeman@stfc.ac.uk

Lecture Outline Introduction FPGA Field Programmable Gate Array 11/6/2017 Introduction Why Programmable Logic Devices and FPGAs FPGA Field Programmable Gate Array Architecture Design Flow Hardware Description Languages Design Tools ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Designing Logic with FPGAs 11/6/2017 Design Capture. High level Description of Logic Design. Graphical descriptions Hardware Description Language (Textual) ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Hardware Description Languages 11/6/2017 Language describing hardware (Engineers call it FIRMWARE) Doesn’t behave like “normal” programming language ‘C/C++’ Describe Logic as collection of Processes operating in Parallel Language Constructs for Synchronous Logic Compiler (Synthesis) Tools recognise certain code constructs and generates appropriate logic Not all constructs can be implemented in FPGA! 2 Popular languages are VHDL , VERILOG Easy to start learning… Hard to master! ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

VHDL ENTITY Declaration Input Output to Module (STD LOGIC) 11/6/2017 VHDL ENTITY Declaration Input Output to Module (STD LOGIC) SIGNALS Declaration WIRES CONCURRENT ASSIGNMENTS CONDITIONAL ASSIGNMENTS => MULTIPLEXERS edward.freeman@stfc.ac.uk

VHDL PROCESS Declaration. CONCURRENT functions. Synchronous Logic. 11/6/2017 VHDL PROCESS Declaration. CONCURRENT functions. Synchronous Logic. COMPONENT Declaration edward.freeman@stfc.ac.uk

Designing Logic with FPGAs 11/6/2017 High level Description of Logic Design Hardware Description Language (Textual) Compile (Synthesis) into NETLIST. Boolean Logic Gates. Target FPGA Device Mapping Routing Bit File for FPGA Commercial CAE Tools (Complex & Expensive) Logic Simulation Design Flow ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time edward.freeman@stfc.ac.uk

Configuring an FPGA 11/6/2017 Millions of SRAM cells holding LUTs and Interconnect Routing Volatile Memory. Lose configuration when board power is turned off. Keep Bit Pattern describing the SRAM cells in non-Volatile Memory e.g. PROM or Digital Camera card Configuration takes ~ secs JTAG Port Programming Bit File ‘outline’ of coverage time-plan - What is an FPGA - And why are they important for HEP questions - any time JTAG Testing edward.freeman@stfc.ac.uk

Not just logic Hard blocks (built into the FPGA) High speed serialises (1Gb, 10Gb, hyper-transport ect) Complex multiplier units (DSP) Embedded processors (PPC404, PPC440, ARM Cortex-A9) PCI express (Gen 2) Multi clock multi phase clock managers. Built in ultra fast RAMs Programmable IO. (LVDS, SSTL and 100’s of others) Plus the millions of gates of programmable logic from the FPGA fabric its self. edward.freeman@stfc.ac.uk

Serialisers A number of types and speeds. We are currently supporting projects with multi 1Gb Ethernet readout. 10Gb is working in the lab and 1st boards are in testing now PCI express endpoint (Gen 1) Camera link Can use off the self switches to make backend system edward.freeman@stfc.ac.uk PCI express

Embedded processors and multipliers Fast image processing and control loop feedback in C code software. (20KHz image rate) Large ping pong (image) frame buffer System monitoring, house keeping, reporting and logging Can also have an army of small “soft” processor cores Multipliers (DSP blocks) Complex FFT for machine frequency control. edward.freeman@stfc.ac.uk

Block RAMs and Clock managers Small high speed buffers Look up tables and scratch pads. FIFO’s to help adjust data rates between processing blocks. Clock Managers Generate different frequency's from a reference clock. Generate phase shits of the clocks. Distribute the clocks to the different areas of the FPGAs (Not of much interest but nothing works without them) edward.freeman@stfc.ac.uk

Other technology you can add to an FPGA Memory interfaces DRAM, DDR, QDR, SRAM, ZBTRAM ect. Industry standard memory modules DDR2, DDR3, Maths functions Floating point units Complex Multiplier Integer Add, Sub, Multiply, Div Digital signal processing functions FFT, FIR, reed-solomon encoders Or any other digital system that can be described with custom logic. edward.freeman@stfc.ac.uk

Field Programmable Gate Arrays FPGA 11/6/2017 Large Complex Functions Re-Programmability, Flexibility. Massively Parallel Architecture Processing many channels simultaneously cf MicroProcessor Fast Turnaround Designs  Standard IC Manufacturing Processes. Moore’s Law  Mass produced. Inexpensive.  Many variants. Sizes. Features.  PP Not Radiation Hard  Power Hungry  No Analogue  A field-programmable gate array (FPGA) is a large-scale integrated circuit that can be programmed after it is manufactured rather than being limited to a predetermined, unchangeable hardware function. The term "field-programmable" refers to the ability to change the operation of the device "in the field," while "gate array" is a somewhat dated reference to the basic internal architecture that makes this after-the-fact reprogramming possible. edward.freeman@stfc.ac.uk

? FPGA Trends State of Art is 32nm on 300 mm wafers 11/6/2017 State of Art is 32nm on 300 mm wafers Top of range >500,000 Logic Blocks >1,000 pins (Fine Pitched BGA) Logic Block cost ~ 1$ in 1990 Today < 0.1 cent Problems Power. Leakage currents. Design Gap CAE Tools If its impossible today it will be hard tomorrow and easy next year time-plan - What is an FPGA - And why are they important for HEP questions - any time ? edward.freeman@stfc.ac.uk

Summary FPGA Field Programmable Gate Arrays Design Flow 11/6/2017 Summary FPGA Field Programmable Gate Arrays Architecture Design Flow Hardware Description Languages Design Tools Exploit industry hardware and protocols Importance for Particle Physics Experiments edward.freeman@stfc.ac.uk

References and contacts 11/6/2017 References and contacts The Design Warrior’s Guide to FPGAs Clive Maxfield, Newnes Elsevier FPGA manufacturer web sites www.xilinx.com www.altera.com FPGA Online www.pldesignline.com www.fpgajournal.com www.doulos.com Technology Rob Halsall – Rob.Halsall@stfc.ac.uk John Coughlan – John.Coughlan@stfc.ac.uk edward.freeman@stfc.ac.uk