Computer Architecture Lecture 8: Memory hierarchy. Cache memory Piotr Bilski

Characteristics of the memory systems Location Capacity Transfer unit Access mode Performance Physical structure Physical characteristics Organization

Memory location Processor (registers, L1cache memory) Internal (main) memory (RAM) External memory (auxilary – disk drives)

Memory capacity Word size Number of words Memory capacity is expressed in bytes and their multiplications, so: 1 B = 8 b 1 KB = 1024 B, 1 MB = 1024 KB etc.

Transfer unit Number of the data lines connected to the memory module (normally equal to the word length), but: –Word is a basic unit in the memory organization –Adressable unit is used to direct memory addressing (byte or word) –Transfer unit can be equal to word or addressable unit

Memory access modes Sequential access (e.g. tape memory) Direct access (disk memory) Random access (main memory) Associative access (cache memory)

Memory performance Access time– time between putting address to the address bus and acquiring information on the data bus Cycle time – access time increased by the time of the gap between the next access Transfer speed – for RAM: 1 / cycle time

Physical memory structure Semiconductor (RAM, ROM) Magnetic (hard disks, floppy disks, streamers) Optical (CD-ROM, DVD-ROM) Magnetooptical (WORM)

Physical characteristics Volatility –Volatile memory (RAM) –Non-volatile memory (ROM) Content modification –Erasable (np. RAM, EPROM) –Non-erasable (ROM)

Memory organization One level („flat”) Multilevel (e.g. cache) Hit ratio 0 1 T2T1T2T1 T 1 + T 2 Access time

Memory hierarchy Processor registers Cache memory Main (operational) memory External memory Capacity Speed Access time cost  access time –  cost / bit  capacity –  cost / bit  capacity –  access time

Why do we need cache memory? Locality of references rule – executed program consists of the fragments existing next to each other and executed one by one Time locality Spatial locality

Cache memory work regime C-1 Block Rows Flag Block length (K words) Memory address Block 1 (K words) Block N (K words) Word length 2 n - 1 Main memory addressed using n bits (total 2 n words) Cache memory has C rows

Cache memory work regime (cont.) Processor Cache memory Main memory Transfer of words Transfer of blocks

Reading from cache memory START Acquiring address from CPU Is this block’s address in the cache memory? EXECUTION Accessing main memory for the addressed block Assignment of the block to the cache memory row Transferring block into the cache memory Transferring word to CPU NO Transfer of word to CPU YES

Details of the cache memory Size Mapping Replacement algorithm Writing algorithm Row size Number of the cache memories

Size of the cache memory Minimization of the memory cost Maximization of the processor’s speed ProcessorType Prod. year L1 cache instruction L1 cache data L2 memory L3 memory IBM 360Mainframe KBNone IBM 3033Mainframe KBNone 80486PC19898 KBNone PentiumPC19938 KB 256/512None PowerPC G4 PC/serv KB 256/1 MB 2 MB Pentium 4PC/serv KB 256 KBNone ItaniumPC/serv KB 96 KB4 MB Athlon XpPC/serv KB 512 KBNone Athlon 64PC/serv KB 1 MBNone

Mapping function The number of the rows in the cache is smaller than the number of the blocks in the main memory Three methods exist: –Direct –Associative –Set-associative

Cache memory with direct mapping Comparison Flag RowWord Memory address Flag … Data L0L0 LiLi Main memory B0B0 W0 W1 W2 W3 s-rr w w s+w s w hit miss

Direct mapping (cont.) i – number of the row in the cache memory j – number of the block in the main memory m – number of rows in the cache memory i = j mod m Address length: s+w bits Number of the addressed units: 2 s+w words Block size = row size: 2 w words Number of blocks in the main memory: 2 s Number of rows in the cache memory: 2 r

Result of the direct mapping Row in the cache memory Assigned blocks in the main memory 00, m, 2m,..., 2 s – m 11, m+1, 2m+1,..., 2 s – m m-1m-1, 2m-1, 3m-1,..., 2 s – 1

Example of the direct mapping For the cache memory having 2 14 rows (4 B each) and main memory of 16 MB capacity: Row in the cache memoryAssigned main memory blocks , ,..., FF , ,..., FF FFFC, 01FFFC,..., FFFFFC Row width: 8 b flag, 32 b data

Cache memory of associative mapping Comparison FlagWord Memory address Flag … Data L0L0 LiLi Main memory B0B0 W0 W1 W2 W3 s w w s+w s w hit miss s s

Associative mapping (cont.) Address length: s+w bits Number of the addressed units: 2 s+w words Block size = row size: 2 w words Number of the main memory blocks: 2 s Number of rows in the cache memory: any Flag size: s words

Example of the associative mapping Address A FFFFF4 FFFFF8 FFFFFC Data F235A72C 3982FB1A Flag Data 22 b 32 b Flag (22 b) Word (2 b) FFFFF 048D5E FB1A F235A72C

Cache memory with set-associative mapping Comparison Flag SectionWord Memory address Flag … Data S0S0 SiSi Main memory W0 W1 W2 W3 s-d d w s+w hit miss

Set-associative mapping (cont.) i – number of the row in the cache memory j – number of the block in the main memory m – number of rows in the cache memory m = v x k i = j mod v Address length: s+w bits Number of addressed units: 2 s+w words Block size = row size: 2 w words Number of blocks in the main memory: 2 s

Set-associative mapping (cont.) Number of rows in a section: k Number of sections: v = 2 d Number of rows in the cache memory: kv = k x 2 d Flag size: (s-d) bits

Example of the set-passociative mapping Flag A 1FF Data F235A72C FB1A Flag Data 9 b 32 b Flag (9 b) Słowo (2 b) A Section (13 b) FFC FFC 9 b 32 b 01A F235A72C

Algorithms of the cache memory content replacement Least recently used (LRU) First in - first out (FIFO) Least frequently used (LFU) Random choice

Algorithms of writing into the cache memory write through write back System assuring consistency (multiprocessor system with cache) –Bus control with write through –Hardware transparency –Memory not mapped by the cache memory

Other problems Row size and block size Number of the cache memories –Memory of the higher level is integrated in one chip with the processor, works with identical frequency –Memory of the lower level works with the bus frequency (it is on the mainboard)

Pentium 4 cache memory

Pentium 4 processor core Instruction fetching/decoding unit –Fetches instructions from L2 cache memory –Decodes them into microoperations –transfers microoperations to L1 cache memory Non-sequential instruction execution unit –Queues microoperations Execution units –Execute microoperations –Fetch data from the L1 cache –Write results into the registers Memory subsystem –Communicates with the system bus and L2 cache memory

PowerPC cache memory ProcessorSizeB / rowOrganization PowerPC 6011 x 32 KB328-way PowerPC 6032 x 8 KB322-way PowerPC 6042 x 16 KB324-way PowerPC 6202 x 32 KB648-way PowerPC G32 x 32 KB648-way PowerPC G42 x 32 KB328-way

PowerPC cache memory (cont.)