1. IRAM and ISTORE Projects
Aaron Brown, James Beck, Rich Fromm, Joe Gebis, Paul Harvey, Adam Janin, Dave Judd, Kimberly Keeton, Christoforos Kozyrakis, David Martin, Rich Martin, Thinh Nguyen, David Oppenheimer, Steve Pope, Randi Thomas, Noah Treuhaft, Sam Williams, John Kubiatowicz, Kathy Yelick, and David Patterson
Fall 2000 DIS DARPA Meeting

2. IRAM and ISTORE Vision
Integrated processor in memory provides efficient access to high memory bandwidth
Two “Post-PC” applications:
IRAM: Single chip system for embedded and portable applications
Target media processing (speech, images, video, audio)
ISTORE: Building block when combined with disk for storage and retrieval servers
Up to 10K nodes in one rack
Non-IRAM prototype addresses key scaling issues: availability, manageability, evolution
Photo from Itsy, Inc.

3. IRAM Overview
A processor architecture for embedded/portable systems running media applications
Based on media processing and embedded DRAM
Simple, scalable, energy and area efficient
Good compiler target
Flag 0
Flag 1
Instr Cache (8KB)
FPU
Flag Register File (512B)
MIPS64™
5Kc Core
CP IF
Arith 0
Arith 1
256b
256b
SysAD IF
I will start with what is interesting about Vector IRAM.
This is a prototype microprocessor that integrates a vector unit with 256 bit datapaths with a 16 MByte embedded DRAM memory system. The design uses 150 million transistors and occupies nearly 300 square mm. While operating at just 200 MHz, Vector IRAM achieves 3.2 giga ops and consumes 2 Watts. Vector IRAM also comes with an industrial strength vectorizing compiler for software development.
Vector IRAM is being implemented by a group of only 6 graduate students, responsible for architecture, design, simulation and testing. So, if Patterson and Hennessy decide to introduce performance/watt/man year as major processor metric in the new version of their book, this processor will likely be one of the best in this class.
Data Cache (8KB)
Vector Register File (8KB)
64b
64b
Memory Unit
TLB
JTAG IF
256b
DMA
Memory Crossbar …
JTAG
DRAM0
(2MB)
DRAM1
(2MB)
DRAM7
(2MB)

4. Architecture Details MIPS64™ 5Kc core (200 MHz) Vector unit (200 MHz)
Single-issue scalar core with 8 Kbyte I&D caches
Vector unit (200 MHz)
8 KByte register file (32 64b elements per register)
256b datapaths, can be subdivided into 16b, 32b, 64b:
2 arithmetic (1 FP, single), 2 flag processing
Memory unit
4 address generators for strided/indexed accesses
Main memory system
8 2-MByte DRAM macros
25ns random access time, 7.5ns page access time
Crossbar interconnect
12.8 GBytes/s peak bandwidth per direction (load/store)
Off-chip interface
2 channel DMA engine and 64n SysAD bus
The vector unit is also connected to the coprocessor interface of the MIPS processor and works at 200 MHz. It includes a multiported 8 Kbyte register file. This allows each of the 32 registers to hold 32 64bit elements or 64 32b elements and so on. The flag register file has capacity of half a Kbyte. There are two functional units for arithmetic operations. Both can executed integer and logical operations, but only one can executed floating-point. There are also 2 flag processing units which provide support for predicated execution and exception handling. Each of the functional units has a 256 bit pipelined datapath. One each cycle, 4 64b operations or 8 32b operations or 16 16b operations can execute in parallel. To simplify the design and reduce area requirements, our prototype does not implement 8b integer operations and double precision arithmetic. All operations excluding divides are fully pipelined.
The vector coprocessor also includes one memory or load/store unit. The LSU can exchange up to 265b per cycle with the memory system and has four address generators for strided and indexed accesses. Address translation is performed in a two level TLB structure. The hardware managed, first level, microTLB has four entries and four ports, while the main TLB has 32 double-page entries and a single access port. The main TLB is managed by software. The memory unit is pipelined and up to 64 independent accesses may be pending at any time.
The 64b SysAD bus connects to external chip-set at 100 MHz

5. Floorplan Technology: IBM SA-27E 0.18mm CMOS, 6 metal layers
290 mm2 die area
225 mm2 for memory/logic
Transistor count: ~150M
Power supply
1.2V for logic, 1.8V for DRAM
Typical power consumption: 2.0 W
0.5 W (scalar) W (vector) W (DRAM) W (misc)
Peak vector performance
1.6/3.2/6.4 Gops wo. multiply-add (64b/32b/16b operations)
3.2/6.4 /12.8 Gops w. madd
1.6 Gflops (single-precision)
Tape-out planned for March ‘01
14.5 mm
20.0 mm
This figure presents the floorplan of Vector IRAM. It occupies nearly 300 square mm and 150 million transistors in a 0.18um CMOS process by IBM. Blue blocks on the floorplan indicate DRAM macros or compiled SRAM blocks. Golden blocks are those designed at Berkeley. They included synthesized logic for control and the FP datapaths, and full custom logic for register files, integer datapaths and DRAM.
Vector IRAM operates at 200MHz. The power supply is 1.2V for logic and 1.8V for DRAM. The peak performance for the vector unit is 1.6 giga ops for 64bit integer operations. Performance doubles or quadruples for 32 and 16b operations respectively. Peak floating point performance is 1.6 Gflops.
There are several interesting things to notice on the floorplan. First the overall design modularity and scalability. It mostly consists of replicated DRAM macros and vector lanes connected through a crossbar. Another very interesting feature is the percentage of this design directly visible to software. Compilers can control any part of the design that is registers, datapaths or main memory. They do that by scheduling proper arithmetic or load store instructions. The majority of our design is used for main memory, vector registers and datapaths. On the other hand, if you take a look at a processor like Pentium 3, you will see that less than 20% of its are is used for datapaths and registers. The rest is caches and dynamic issue logic. While this usually work for the benefit of applications, they cannot be controlled by compiler and they cannot be turned off when not necessary.

6. Alternative Floorplans
“VIRAM-8MB”
4 lanes, 8 Mbytes
190 mm2
3.2 Gops at 200 MHz (32-bit ops)
“VIRAM-2Lanes”
2 lanes, 4 Mbytes
120 mm2
1.6 Gops at 200 MHz
“VIRAM-Lite”
1 lane, 2 Mbytes
60 mm2
0.8 Gops at 200 MHz

7. VIRAM Compiler Frontends Optimizer Code Generators C T3D/T3E Cray’s
PDGCS
C++
C90/T90/SV1
Fortran95
SV2/VIRAM
Based on the Cray’s production compiler
Challenges:
narrow data types and scalar/vector memory consistency
Advantages relative to media-extensions:
powerful addressing modes and ISA independent of datapath width
Apart from the hardware, we have also worked on software development tools. We have a vectorizing compiler with C, C++, and Fortran front-ends. It is based on the production compiler by Cray for its vector supercomputers, which we ported to our architecture. Its has extensive vectorization capabilities including outer-loop vectorization. Using this compiler, vectorize applications written in high level languages without necessarily using optimized libraries or “special” (non-standard) variable types in his application.

8. Exploiting 0n-Chip Bandwidth
Vector ISA uses high bandwidth to mask latency
Compiled matrix-vector multiplication: 2 Flops/element
Easy compilation problem; stresses memory bandwidth
Compare to 304 Mflops (64-bit) for Power3 (hand-coded)
Performance scales with number of lanes up to 4
Need more memory banks than default DRAM macro for 8 lanes
The IBM Power 3 number is from the latest LAPACK manual, and is for the BLAS2 (dgemv) performance, presumably hand-coded by IBM experts.
The IRAM numbers are from the Cray compiler. This algorithm requires either strided access or reduction operations. The compiler uses the strided accesses. (Reductions are worse, because more time is spent with short vectors.)
Because of the strided accesses, we start to have bank conflicts with more lanes. I think we had trouble getting the simulator to do anything reasonable with subbanks, so this reports 16 banks, rather than 8 banks with 2 subbanks per bank.
The BLAS numbers for the IBM are better than for most other machines without such expensive memory systems. E.g., the Pentium III is 141, the SGI O2K is 216, the Alpha Miata is 66, and the Sun Enterprise is 450 is 267. Only the a AlphaServer DS-20 at 372 is beats VIRAM-1 (4 lanes, 8 banks) at 312.
None of the IRAM number use a multiply add – performance would definitely increase with that.

9. Compiling Media Kernels on IRAM
The compiler generates code for narrow data widths, e.g., 16-bit integer
Compilation model is simple, more scalable (across generations) than MMX, VIS, etc.
Strided and indexed loads/stores simpler than pack/unpack
Maximum vector length is longer than datapath width (256 bits); all lane scalings done with single executable
The IBM Power 3 number is from the latest LAPACK manual, and is for the BLAS2 (dgemv) performance, presumably hand-coded by IBM experts.
The IRAM numbers are from the Cray compiler. This algorithm requires either strided access or reduction operations. The compiler uses the strided accesses. (Reductions are worse, because more time is spent with short vectors.)
Because of the strided accesses, we start to have bank conflicts with more lanes. I think we had trouble getting the simulator to do anything reasonable with subbanks, so this reports 16 banks, rather than 8 banks with 2 subbanks per bank.
The BLAS numbers for the IBM are better than for most other machines without such expensive memory systems. E.g., the Pentium III is 141, the SGI O2K is 216, the Alpha Miata is 66, and the Sun Enterprise is 450 is 267. Only the a AlphaServer DS-20 at 372 is beats VIRAM-1 (4 lanes, 8 banks) at 312.
None of the IRAM number use a multiply add – performance would definitely increase with that.

10. IRAM Status Chip Compiler Application & Benchmarks
ISA has not changed significantly in over a year
Verilog complete, except SRAM for scalar cache
Testing framework in place
Compiler
Backend code generation complete
Continued performance improvements, especially for narrow data widths
Application & Benchmarks
Handcoded kernels better than MMX,VIS, gp DSPs
DCT, FFT, MVM, convolution, image composition,…
Compiled kernels demonstrate ISA advantages
MVM, sparse MVM, decrypt, image composition,…
Full applications: H263 encoding (done), speech (underway)
To conclude my talk, today I have presented to you Vector IRAM. This is an integrated architecture for media processing that combines a 256 bit vector unit with 16 Mbytes of embedded DRAM. It uses 150 million transistors and 300 square mm. At just 200 MHz, it achieves 3.2 giga ops for 32b integers and consumes 2 Watts. It is a simple, scalable design that is efficient in terms of performance, power, and area.
The current status of the prototype design is the following. We are currently in the verification and back-end stage of the design. RTL development and the design of several full custom components has been completed. We expect to tape-out the design by the end of the year. The compiler is also operational and is being tuned for performance. We are also working on applications for this system.

11. Scaling to 10K Processors
IRAM + micro-disk offer huge scaling opportunities
Still many hard system problems (AME)
Availability
systems should continue to meet quality of service goals despite hardware and software failures
Maintainability
systems should require only minimal ongoing human administration, regardless of scale or complexity
Evolutionary Growth
systems should evolve gracefully in terms of performance, maintainability, and availability as they are grown/upgraded/expanded
These are problems at today’s scales, and will only get worse as systems grow

12. Is Maintenance the Key? Rule of Thumb: Maintenance 10X HW
so over 5 year product life, ~ 95% of cost is maintenance
VAX crashes ‘85, ‘93 [Murp95]; extrap. to ‘01
HW/OS 70% in ‘85 to 28% in ‘93. In ‘01, 10%?
[Murp95] Murphy, B.; Gent, T. Measuring system and software reliability using an automated data collection process. Quality and Reliability Engineering International, vol.11, (no.5), Sept.-Oct p

13. Hardware Techniques for AME
Cluster of Storage Oriented Nodes (SON)
Scalable, tolerates partial failures, automatic redundancy
Heavily instrumented hardware
Sensors for temp, vibration, humidity, power, intrusion
Independent diagnostic processor on each node
Remote control of power; collects environmental data for
Diagnostic processors connected via independent network
On-demand network partitioning/isolation
Allows testing, repair of online system
Managed by diagnostic processor
Built-in fault injection capabilities
Used for hardware introspection
Important for AME benchmarking

14. Storage-Oriented Node “Brick”
ISTORE-1 system
Hardware: plug-and-play intelligent devices with self-monitoring, diagnostics, and fault injection hardware
intelligence used to collect and filter monitoring data
diagnostics and fault injection enhance robustness
networked to create a scalable shared-nothing cluster
Scheduled for 4Q 00
ISTORE Chassis
80 nodes, 8 per tray
2 levels of switches:
Mb/s
2 1 Gb/s
Environment Monitoring:
UPS, redundant PS,
fans, heat and vibration sensors...
Storage-Oriented Node “Brick”
Portable PC CPU: Pentium II/266 + DRAM
Redundant NICs (4 100 Mb/s links)
Diagnostic Processor
Disk
Half-height canister
-more CPU per node than most existing NAS devices, fewer disks per node than most clusters
- gives us research flexibility: runs existing cluster apps while leaving lots of free CPU for introspection

15. ISTORE-1 System Layout PE1000s PE1000s: PowerEngines 100Mb switches
PE5200s: PowerEngines 1 Gb switches
UPSs: “used”
Patch panel
Patch panel
PE5200
Brick shelf
Brick shelf
Brick shelf
Brick shelf
Brick shelf
Brick shelf
Brick shelf
Brick shelf
Brick shelf
Brick shelf
Patch panel
Patch panel
PE5200
UPS
UPS
UPS
UPS
UPS
UPS

16. ISTORE Brick Node Block Diagram
Mobile Pentium II Module
SCSI
North
Bridge
CPU
Disk (18 GB)
South
Bridge
Diagnostic
Net
DUAL
UART
DRAM
256 MB
Super
I/O
Monitor
&
Control
Diagnostic
Processor
BIOS
Ethernets
4x100 Mb/s
PCI
Sensors for heat and vibration
Control over power to individual nodes
Flash
RTC
RAM

17. ISTORE Brick Node
Pentium-II/266MHz
256 MB DRAM
18 GB SCSI (or IDE) disk
4x100Mb Ethernet
m68k diagnostic processor & CAN diagnostic network
Packaged in standard half-height RAID array canister

18. Software Techniques Reactive introspection
“Mining” available system data
Proactive introspection
Isolation + fault insertion => test recovery code
Semantic redundancy
Use of coding and application-specific checkpoints
Self-Scrubbing data structures
Check (and repair?) complex distributed structures
Load adaptation for performance faults
Dynamic load balancing for “regular” computations
Benchmarking
Define quantitative evaluations for AME

19. Network Redundancy Each brick node has 4 100Mb ethernets
TCP striping used for performance
Demonstration on 2-node prototype using 3 links
When a link fails, packets on that link are dropped
Nodes detect failures using independent pings
More scalable approach being developed
Mb/s

20. Load Balancing for Performance Faults
Failure is not always a discrete property
Some fraction of components may fail
Some components may perform poorly
Graph shows effect of “Graduated Declustering” on cluster I/O with disk performance faults

21. Availability benchmarks
Goal: quantify variation in QoS as fault events occur
Leverage existing performance benchmarks
to generate fair workloads
to measure & trace quality of service metrics
Use fault injection to compromise system
Results are most accessible graphically

22. Example: Faults in Software RAID
Linux
Solaris
Compares Linux and Solaris reconstruction
Linux: minimal performance impact but longer window of vulnerability to second fault
Solaris: large perf. impact but restores redundancy fast

23. Towards Manageability Benchmarks
Goal is to gain experience with a small piece of the problem
can we measure the time and learning-curve costs for one task?
Task: handling disk failure in RAID system
includes detection and repair
Same test systems as availability case study
Windows 2000/IIS, Linux/Apache, Solaris/Apache
Five test subjects and fixed training session
(Too small to draw statistical conclusions)

24. Sample results: time
Graphs plot human time, excluding wait time

25. Analysis of time results
Rapid convergence across all OSs/subjects
despite high initial variability
final plateau defines “minimum” time for task
plateau invariant over individuals/approaches
Clear differences in plateaus between OSs
Solaris < Windows < Linux
note: statistically dubious conclusion given sample size!

26. ISTORE Status ISTORE Hardware All 80 Nodes (boards) manufactured
PCB backplane: in layout
Finish 80 node system: December 2000
Software
2-node system running -- boots OS
Diagnostic Processor SW and device driver done
Network striping done; fault adaptation ongoing
Load balancing for performance heterogeneity done
Benchmarking
Availability benchmark example complete
Initial maintainability benchmark complete, revised strategy underway

27. BACKUP SLIDES
IRAM

28. IRAM Latency Advantage
1997 estimate: 5-10x improvement
No parallel DRAMs, memory controller, bus to turn around, SIMM module, pins…
30ns for IRAM (or much lower with DRAM redesign)
Compare to Alpha 600: 180 ns for 128b; 270 ns for 512b
2000 estimate: 5x improvement
IRAM memory latency is 25 ns for 256 bits, fixed pipeline delay
Alpha 4000/4100: 120 ns
1st
2nd innovate inside DRAM
Even compared to latest Alpha

29. IRAM Bandwidth Advantage
1997 estimate: 100x
1024 1Mbit modules, each 1Kb wide(1Gb chip)
40 ns RAS/CAS = 320 GBytes/sec
If cross bar switch or multiple busses deliver 1/3 to 2/3 of total Þ GBytes/sec
Compare to: AlphaServer 8400 = 1.2 GBytes/sec, =1.1 Gbytes/sec
2000 estimate: x
VIRAM-1 16 MB chip divided into 8 banks => 51.2 GB peak from memory banks
Crossbar can consume 12.8 GB/s
6.4GB/s from Vector Unit GB/s from either scalar or I/O
2nd reason
Delivered BW on Alpha Server

30. Power and Energy Advantages
1997 Case study of StrongARM memory hierarchy vs. IRAM memory hierarchy
cell size advantages Þ much larger cache Þ fewer off-chip references Þ up to 2X-4X energy efficiency for memory-intensive algorithms
less energy per bit access for DRAM
Power target for VIRAM-1
2 watt goal
Based on preliminary spice runs, this looks very feasible today
Scalar core included
bigger caches or less memory on board
Cache in logic process vs.
SRAM in SRAM process vs.
DRAM in DRAM process
Main reason

31. Summary IRAM takes advantage of high on-chip bandwidth
Vector IRAM ISA utilizes this bandwidth
Unit, strided, and indexed memory access patterns supported
Exploits fine-grained parallelism, even with pointer chasing
Compiler
Well-understood compiler model, semi-automatic
Still some work on code generation quality
Application benchmarks
Compiled and hand-coded
Include FFT, SVD, MVM, sparse MVM, and other kernels used in image and signal processing