1. Lecture 2: Basic Notions and Fundamentals
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

2. Outline Basic computer organization Architectural drivers Measures
von Neumann model and execution cycle
Pipelines and caches
Architectural drivers
Technology
Applications and compatibility
Compilers
Measures
Methodology
Key measures
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

3. von Neumann’s Contribution
Instruction Cycle
Memory
Control
Fetch
Decode
Evaluate addresses
Fetch operands
Execute
Store results
program …
Datapath
Input/
Output
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

4. Pipelining Instruction latency does not decrease Throughput increases
Fetch
Decode
Execute
Mem
Write
Instruction latency does not decrease
Throughput increases
Dependencies degrade performance
General trends: deeper/wider until 2004
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

5. Multiple Issue Decode/Swap Fetch adder Mem write ALU write ALU writeBR
Stall
logic
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

6. Pipeline of the Pentium IV [Courtesy of Intel Corp]1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
TC nxt IP
TC Fetch
Drv
Alloc
Rename
Que
Sch
Sch
Sch
Disp
Disp RF RF Ex
Flgs
BrCk
Drv
Enables very high clock rate
Enables scalability from one technology to the next
But does it always lead to highest performance?
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

7. Speeding up memory
Fundamental: Principles of locality of memory references
Reference to X  another reference to X later
Reference to X  reference to Y, where X, Y are close
Fundamental: Memory tends to be small/fast/expensive or large/slow/cheap
Result: Memory hierarchies with caching
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

8. Basic Cache Organization
Address
tag
index
offset
Direct Mapped Cache
tag
valid
cache line …
hit/miss
Data
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

9. Cross-section of inverter in
CMOS Inverter
gate
polysilicon
gate
Vdd
gate oxide
p-MOS trans
field oxide
metal
Vdd
Vss p+ p+ n+ n+
Input
Output
n well
p substrate
p-MOS trans
n-MOS trans
n-MOS trans
gate
Vss
Cross-section of inverter in
n-well process
Schematic Notation
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

10. CMOS Trends [Collated from the 2000 update of the International Technology Roadmap for Semiconductors]
Year
1995
1998
2001
2004
2007
2010
2013
Feature Size
(nanometers)
350nm
180nm
130nm
90nm
65nm
45nm
33nm
Transistor
Count
10M
50M
110M
350M
1300M
3500M
11000M
For high-performance CPUs, the feature size
is (typically) the minimum width of a gate n+ n+
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

11. An Microarchitect’s Model of CMOS Power and Delay
Vdd
p-MOS trans
Delay is proportional to the number of gates
typical measure FO4
Power dissipation
dynamic (switching)
Static (leakage)
Input
Output
n-MOS trans
Vss
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

12. CMOS Trends [Collated/extrapolated from the 2000 update of the International Technology Roadmap for Semiconductors]
Year
1995
1998
2001
2004
2007
2010
2013
Feature Size
(nanometers)
350nm
180nm
130nm
90nm
65nm
45nm
33nm
Transistor
Count
10M
50M
110M
350M
1300M
3500M
11000M
Projected FO4 delays
In each pipe satge 27 13 11 8 7 6
This will likely be revised due to the transistor variability problem from 65nm generation on.
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

13. CMOS Trends Transistors become more plentiful but variable
Gates become faster
Wires become relatively slower
Memory becomes relatively slower
Power becomes a critical issue
Noise, error rates, design complexity
CMOS issues
-- Why do wires become slower?
Derivative issues
Memory – board-level signaling, access time, etc
Power – peak power density, energy, dynamic and static
-- notion of a limiting factor (peak power prevents integration)
Error rates
-- small dimensions, close lines
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

14. RC delay of 1mm interconnect
Trends in hardware
High variability
Increasing speed and power variability of transistors
Limited frequency increase
Reliability / verification challenges
Large interconnect delay
Increasing interconnect delay and shrinking clock domains
Limited size of individual computing engines
130nm
30% 5X
0.9
1.0
1.1
1.2
1.3
1.4 1 2 3 4 5
Normalized Leakage (Isb)
Normalized Frequency
Interconnect RC Delay 1 10
100
1000
10000
350
250
180
130 90 65
Delay (ps)
Clock Period
RC delay of 1mm interconnect
Copper Interconnect
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois
Source: Shekhar Borkar, Intel

15. Dynamic-Static Interface
Moving functionality into compilers
Most architectures today, including X86 rely on compilers to achieve performance goals
A major issue is the number of bits required to deliver information from the compiler to the runtime hardware
Highly optimizing compiler also reduced the incentive for machine language programming, which makes portable programs a reality.
The more implementation details one exposes in the instruction set architecture, the more difficult it is to adopt new implementation techniques.
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

16. Applications Applications drive architecture from ‘above”
Designing next-generation computers involves understanding the behavior of applications of importance and exploiting their characteristics
What are applications of importance?
For us, we will often use benchmarks to characterize different architectural options
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

17. In the image of the simple hardware, humans created complex software
Software creation based on a simple execution model
Instructions are considered to execute sequentially
Data objects are mapped into a flat, monolithic store reachable by all
Reality when laid out by von Neumann in the 40’s; abstraction now
This execution abstraction has been used in development of large, complex software
“Traditional software model”
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

18. Future apps reflect a concurrent world
Exciting applications in future mass computing market have been traditionally considered “supercomputing applications”
Physiological simulation – cellular pathways (GE Research)
Molecular dynamics simulation (NAMD at UIUC)
Video and audio coding and manipulation – MPEG-4 (NCTU)
Medical imaging – CT (UIUC)
Consumer game and virtual reality products
These “Super-applications” represent and model physical world
Various granularities of parallelism exist, but…
programming model must support required dimensions
data delivery needs careful management
Do not go over all of the different applications. Let them read them instead.
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

19. Direction of computer architecture
Current general purpose architectures cover traditional applications
New parallel-privatized architectures cover some super-applications
Attempts to grow current architectures “out” or domain-specific architectures “in” lack success
By properly exploiting parallelism of super-applications, the coverage of domain-specific architectures can be extended
This leads to memory wall: Why we can’t transparently extend out the current model to the future application space: memory wall. How does the meaning of the memory wall change as we transition into architectures that target the super-application space.
Lessons learned from Itanium
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

20. Compatibility The case of workstations and servers
Relatively open to new architectures.
Performance is a major concern.
Linux/UNIX is a portable operating system.
Current economics model works against new architectures
The case of personal computers
Very tough on new architectures.
Windows and Apple OS are not portable
Most applications are distributed in binary code
Price is of more concern than performance.
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

21. Experimental Methodology
Selected real programs by characterizing workload
PERFECT Club, SPEC, MediaBench
What do these programs do?
What input was given to these programs?
How are they related to your own workload?
What do experimental results mean?
Require high quality software support.
Tremendous variation in capability
Trace driven simulation vs. re-compilation
Nothing can replace real-machine measurements
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois

22. Measures Iron Law: performance = 1 / execution time
= 1/ (CPI * insts * 1/freq)
(Basis of SPEC Marks)
or (IPC * freq)/insts
CPI : Cycles per instruction (how is this calculated)?
IPC : Instructions per cycle
other useful measures : average memory access time
© Wen-mei Hwu and S. J. Patel, 2005
ECE 511, University of Illinois