Memory It’s all about storing bits--binary digits Vacuum tubes, CRTs, drums, disks, core, ICs Issues of size, cost, speed Semiconductor memories (chips)

Memory How to store How to organize--so as to be able to “store” a bit (or byte or word) and then find it again How to associate an address with a “set” of bits

Up to 2 k addressable locations Memory Processor k-bit address bus MAR Up to 2 k addressable locations n-bit data bus MDR Word length = n bits Control lines ( , MFC, etc.) R / W Figure 5.1. Connection of the memory to the processor.

Memory address MAR MAR bus memory READ bus memory bus data Memory access (read) address MAR MAR bus memory READ bus memory bus data processor bus MFC MDR bus

Memory Memory access time: Time from Read issued to MFC received Memory cycle time: Time between two successive reads

Memory Obviously, the speed of the processor depends on the speed of the memory Random Access Memory (RAM) simply means that access time is fixed (and the same) for all memory locations (addresses)

small (very) example (128 bit chip) ¢ b b ¢ b b ¢ 7 7 1 1 W • FF FF A W • 1 A 1 Address • • • • • • Memory decoder cells A 2 A 3 W • 15 Sense / Write Sense / Write Sense / Write R / W circuit circuit circuit CS Data input/output lines: b b b 7 1 Figure 5.2. Organization of bit cells in a memory chip. small (very) example (128 bit chip) 16 words of 8 bits each (16 x 8)

Figure 5.2. Organization of bit cells in a memory chip. ¢ b b ¢ b b ¢ 7 7 1 1 W • FF FF A W • 1 A 1 Address • • • • • • Memory decoder cells A 2 A 3 W • 15 Sense / Write Sense / Write Sense / Write R / W circuit circuit circuit CS Data input/output lines: b b b 7 1 Figure 5.2. Organization of bit cells in a memory chip.

four input, sixteen output decoder 4 address lines, 16 word addresses A W 1 A 1 Address word lines decoder • A 2 A W 3 15 four input, sixteen output decoder 4 address lines, 16 word addresses

bit 7 of the selected word bit lines b b 7 7 • W . . memory cell (one bit) • W 1 word lines • • . . • W 15 R of course chip select for multi-chip memory / W Sense / Write circuit CS data output lines bit 7 of the selected word b 7

. . . . . . . . bit lines bit lines b b b b 7 7 • • W • • W 1 • • W . . . . • • W 1 word lines • • • • . . . . • • W 15 Sense / Write Sense / Write circuit circuit data output lines b b 7

small (very) example (128 bit chip) ¢ b b ¢ b b ¢ 7 7 1 1 W • FF FF A W • 1 A 1 Address • • • • • • Memory decoder cells A 2 A 3 W • 15 Sense / Write Sense / Write Sense / Write R / W circuit circuit circuit CS Data input/output lines: b b b 7 1 Figure 5.2. Organization of bit cells in a memory chip. small (very) example (128 bit chip) 16 words of 8 bits each (16 x 8)

128 bit chip, 16 words of 8 bits each (16 x 8) ¢ b b ¢ b b ¢ 7 7 1 1 W • FF FF A W • 1 A 1 Address • • • • • • Memory decoder cells A 2 A 3 W • 15 Sense / Write Sense / Write Sense / Write R / W circuit circuit circuit CS Data input/output lines: b b b 7 1 128 bit chip, 16 words of 8 bits each (16 x 8) 4 address lines 8 data lines R / W line chip select line power ground 16 external connections

4 more address lines, total of 20 external connections Chip with 1024 Memory Cells Could be 128 x 8 Same as the 16 x 8, but with more word lines W A W 1 A 1 Address word lines decoder • • A W 7 127 8 data lines 4 more address lines, total of 20 external connections

Chip with 1024 Memory Cells Or, it could be 1024 x 1 W A W 1 A 1 A W 1 A 1 Address word lines (1 bit each) decoder • • A W 9 1023 1 data line 2 more address lines, but only 1 data line, total of 15 external connections

Data input/output (1 bit) 5-bit row W address W 1 32 x 32 memory cell array 5-bit decoder Sense/Write circuitry W 31 10-bit address 32-to-1 R / W output multiplexer and CS input demultiplexer 5-bit column address Data input/output (1 bit) Figure 5.3. Organization of a 1K  1 memory chip.

Multiplexer 2 lines to choose one of the 4 inputs as its output

2 lines to choose one of the 4 inputs as its output

Data input/output (1 bit) 5-bit row W address W 1 32 x 32 memory cell array 5-bit decoder Sense/Write circuitry W 31 10-bit address 32-to-1 R / W output multiplexer and CS input demultiplexer 5-bit column address Data input/output (1 bit) Figure 5.3. Organization of a 1K  1 memory chip.

Static Memory Static: retains its state (content) as long as power is applied and, of course, loses it if powered off (volatile) SRAM: static ram fast expensive

Figure 5.4. A static RAM cell. b b T T 1 2 X Y latch Word line Bit lines Figure 5.4. A static RAM cell.

Transistors in the circuit are effectively switches Voltage = 0 (ground), open switch Voltage = Vs, closed switch

If the cell represents a 1, for example to sense/write circuit b b If the Word line goes high (read), then b = 1, (high) b = 0, (low) sense line sets output high 1 T T 1 2 X Y latch If the Word line is low, nothing on the bit lines Word line Bit lines to sense/write circuit Figure 5.4. A static RAM cell.

If the cell represents a 1, for example to sense/write circuit b b 1 T T To write (say 0), put b low, b high, set Word line high, latch changes 1 2 Then, if the Word line goes high (read) b = 0, b = 1 X Y latch Word line Bit lines to sense/write circuit Figure 5.4. A static RAM cell.

CMOS SRAM Complementary Metal Oxide Semiconductor Uses both “P type” and “N type” transistors

(a) An inverter circuit. V V V R R R V gate V gate V drain drain S V T supply supply supply R R R V gate V gate V out out out drain drain S V T V T in in source source (a) (b) (c) NMOS--closed when Vin raised PMOS--open when Vin raised An inverter circuit.

Figure 5.5. An example of a CMOS memory cell. b b V supply T T 3 4 T T 1 2 X Y T T 5 6 Word line Bit lines Figure 5.5. An example of a CMOS memory cell.

CMOS SRAM Volatile Low power consumption--no current flows except when being accessed Fast--access times of a few nanoseconds Expensive (6 transistors per cell)

Dynamic Ram (DRAM) Simpler cells, higher density 1 million to 16 million bits or more per chip Less expensive But, DRAM cells do not retain their state Must be refreshed periodically

Figure 5.6. A single-transistor dynamic memory cell Bit line Word line capacitor is charged to write a 1 (voltage applied to Word line and to the Bit line) charge on the capacitor will discharge over time T C Figure 5.6. A single-transistor dynamic memory cell

Figure 5.6. A single-transistor dynamic memory cell Bit line Word line If a Read detects a voltage on the capacitor above the “threshold, ” it “sees” a 1, and drives the bit line to full voltage and recharges the capacitor T C If a Read detects a voltage on the capacitor below the “threshold, ” it “sees” a 0, and drives the bit line to ground and fully discharges the capacitor Refreshes whenever read Refresh circuit will periodically read all cells Figure 5.6. A single-transistor dynamic memory cell

16 megabits, 2 million bytes 4096 lines R A S 12 bits to select one of the 4096 rows Row address latch Row decoder 4096 x (512 x 8) cell array 4096 lines / A A CS 20 - 9 8 - Sense / Write circuits R / W 9 bits to select one of the 512 bytes in a row Column address latch Column decoder C A S D D 7 the selected byte Figure 5.7. Internal organization of a 2M x 8 dynamic memory chip. 16 megabits, 2 million bytes

12 bit row address applied, latched on RAS Row Address Strobe 1 R A S 12 bit row address applied, latched on RAS Row address latch Row decoder 4096 x (512 x 8) cell array 21 bit address on 12 lines (reduces external connections) Sense / Write circuits CS R / W Column address latch Column decoder 2 9 bit column address applied, latched on CAS C A S D D 7 Column Address Strobe the selected byte

DRAM Possible to leave row selected (all 512 bytes on sense lines) Then rapidly retrieve successive bytes by changing column addresses Result is a “fast page mode” for “blocks” or “pages” of bytes where appropriate (such as cache loading, disk transfer) Or, synchronous DRAM, SDRAM

SDRAM Can operate in different modes “Burst” modes of different lengths Can transfer “blocks” of data on single Read or Write

Read/Write circuits & latches Mode register and timing control Refresh counter Row address latch Row decoder Cell array Row/Column address Entire row can be addressed and put into latches Successive columns put into output register on successive clock pulses Read/Write circuits & latches Column address counter Columndecoder Clock Mode register and timing control R A S Data input register Data output register C A S R / W C S Clock pulses cause “counting” to select successive columns Data Figure 5.8. Synchronous DRAM.

Figure 5.9. Burst read of length 4 in an SDRAM. Clock R / W R A S C A S Address Row Col Data D0 D1 D2 D3 column address automatically incremented by memory control each cycle row address latched column address latched 2 cycles to activate selected row 1 cycle to put data on data lines Figure 5.9. Burst read of length 4 in an SDRAM.

Larger Memories Using Multiple Chips 512K x 8 memory chip 19 bit address on chip 8-bit data input/output chip select (2 bits) 4 chips 2 million 32-bit words 21 bit address

19-bit internal chip address A 21-bit addresses 18 A 19 A 20 16 chips 2-bit decoder 512K x 8 memory chip D D D D 31-24 23-16 15-8 7-0 Figure 5.10. Organization of a 2M  32 memory module using 512K  8 static memory chips (16 chips).

19-bit internal chip address 4 chips for each 32 bit word A 18 A D D D D 31-24 23-16 15-8 7-0 512K x 8 memory chip Figure 5.10. Organization of a 2M  32 memory module using 512K  8 static memory chips (16 chips).

19-bit internal chip address A 21-bit addresses 18 A 19 A 20 16 chips 2-bit decoder 512K x 8 memory chip D D D D 31-24 23-16 15-8 7-0 Figure 5.10. Organization of a 2M  32 memory module using 512K  8 static memory chips (16 chips).

Figure 5.11. Use of a memory controller. Memory controller does the multiplexing of row and column and issues strobe signals Processor sends all bits of address Row/Column address Address R A S R / W C A S Memory Request controller R / W Memory Processor CS Clock Clock Data Figure 5.11. Use of a memory controller.

Figure 5.11. Use of a memory controller. Row/Column address Address R A S R / W C A S Memory Request controller R / W Memory Processor CS Clock Clock Data Memory controller provides the refresh control if not done on the chip Refreshing typically once every 64 ms. At a cost of .2ms Less than .4% overhead Figure 5.11. Use of a memory controller.

ROM: Read Only Memory Figure 5.12. A ROM cell. Bit line Word line T Not connected to store a 1 Connected to store a 0 P Figure 5.12. A ROM cell.

PROM: Programmable Read Only Memory Bit line Word line Manufactured connected (storing 0), but the connection is a “fuse” and can be burned out with a high current to change it to a 1 T P A PROM cell.

EPROM: Erasable Read Only Memory Bit line Word line Transistor can have a charge put into it that causes it to remain permanently open (programmed to be a 1) Can be erased with ultraviolet light T P Connection to ground always made An EPROM cell.

EEPROM: Electrically Erasable PROM Cells erasable selectively vs. EPROM, erase all Flash Memory Similar to EEPROM--each cell a single transistor with a “trapped” charge Read individual cells, write in blocks Greater density, low power consumption, small, cheap Can substitute for disks (up to a gigabyte?) higher cost, but portable

Figure 5.13. Memory hierarchy. Processor Registers Increasing size Increasing speed Increasing cost per bit Primary cache L1 Secondary cache L2 Main memory Magnetic disk secondary memory Figure 5.13. Memory hierarchy.

Magnetic disk secondary memory Processor always on the processor chip Registers Increasing size Primary cache L1 cache usually SRAM--faster but more expensive Secondary cache L2 may also be on the processor chip main usually DRAM--cheap enough to be large Main memory Magnetic disk secondary memory Figure 5.13. Memory hierarchy.

Cache Memories Main memory (still) slow in comparison to processor speed Main memory constrained by packaging, electronic characteristics and costs Cache memory on the processor chip typically ten times faster than main memory

Locality of Reference Programs tend to spend their time “focused” on particular groups of instructions Loops Frequently called procedures “Localized” areas of programs executed repeatedly during some time period Much (most?) of program not accessed during some time period

Locality of Reference Temporal Recently executed instruction likely to repeat soon When first accessed, move to cache where it will be when referenced again Spatial Instructions near an executed instruction likely to be executed soon When fetching an instruction from memory, move its neighbors into cache as well

Figure 5.14. Use of a cache memory. blocks of memory transferred to (and from) cache Main Processor Cache memory processor accesses instructions and data in the cache if there (a “hit”), in main memory if not (a “miss”) Figure 5.14. Use of a cache memory.

Writing to Cache Write through Cache copy and main memory copy updated simultaneously May repeatedly update the same word in main memory unnecessarily Write back Update cache only Mark cache block “dirty” or “modified” Copy it back to main memory when another block needs the cache space

Cache Management Mapping Determination of where in the cache the blocks (cache lines) of main memory are to be placed Replacement Determination of when to replace a block in cache with another block of main memory Coherency Assurance that no problems arise from cache version differing from main memory version

Direct Mapping Example 64K main memory 16 bit address (word addressed only) View as 4096 blocks of 16 words each Cache of 128 blocks of 16 words

0-31 which block “assigned” to this position is in the cache 0--127 which block in cache 0-15 which word in block The 16-bit address

Direct Mapping Example Main memory blocks 0, 128, 256, etc to block 0 of cache Main memory blocks 1, 129, 257, etc to block 0 of cache

Figure 5.16. Associative-mapped cache. Any block of main memory can be put in any block in the cache Tags are searched “associatively” to find the referenced block Main memory Block 0 Cache Block 1 tag Block 0 tag Block 1 Block i tag Block 127 Tag Word Block 4095 12 4 Main memory address Figure 5.16. Associative-mapped cache.

Figure 5.24. Caches and external connections in Pentium III processor. Processing units L1 instruction cache L1 data cache Bus interface unit System bus Cache bus Main L2 cache Input/Output memory Figure 5.24. Caches and external connections in Pentium III processor.

Figure 5.25. Addressing multiple-module memory systems. k bits m bits Module Address in module MM address ABR DBR ABR DBR ABR DBR Module Module Module i n - 1 (a) Consecutive words in a module m bits k bits Address in module Module MM address ABR DBR ABR DBR ABR DBR Module Module Module i 2 k - 1 (b) Consecutive words in consecutive modules Figure 5.25. Addressing multiple-module memory systems.

Figure 5.26. Virtual memory organization. Processor Virtual address Data MMU Physical address Cache Data Physical address Main memory DMA transfer Disk storage Figure 5.26. Virtual memory organization.