1. Computer Architecture Lecture 8: Memory hierarchy. Cache memory Piotr Bilski

2. Characteristics of the memory systems Location Capacity Transfer unit Access mode Performance Physical structure Physical characteristics Organization

3. Memory location Processor (registers, L1cache memory) Internal (main) memory (RAM) External memory (auxilary – disk drives)

4. Memory capacity Word size Number of words Memory capacity is expressed in bytes and their multiplications, so: 1 B = 8 b 1 KB = 1024 B, 1 MB = 1024 KB etc.

5. Transfer unit Number of the data lines connected to the memory module (normally equal to the word length), but: –Word is a basic unit in the memory organization –Adressable unit is used to direct memory addressing (byte or word) –Transfer unit can be equal to word or addressable unit

6. Memory access modes Sequential access (e.g. tape memory) Direct access (disk memory) Random access (main memory) Associative access (cache memory)

7. Memory performance Access time– time between putting address to the address bus and acquiring information on the data bus Cycle time – access time increased by the time of the gap between the next access Transfer speed – for RAM: 1 / cycle time

8. Physical memory structure Semiconductor (RAM, ROM) Magnetic (hard disks, floppy disks, streamers) Optical (CD-ROM, DVD-ROM) Magnetooptical (WORM)

9. Physical characteristics Volatility –Volatile memory (RAM) –Non-volatile memory (ROM) Content modification –Erasable (np. RAM, EPROM) –Non-erasable (ROM)

10. Memory organization One level („flat”) Multilevel (e.g. cache) Hit ratio 0 1 T2T1T2T1 T 1 + T 2 Access time

11. Memory hierarchy Processor registers Cache memory Main (operational) memory External memory Capacity Speed Access time cost  access time –  cost / bit  capacity –  cost / bit  capacity –  access time

12. Why do we need cache memory? Locality of references rule – executed program consists of the fragments existing next to each other and executed one by one Time locality Spatial locality

13. Cache memory work regime 0 1 2 C-1 Block Rows Flag Block length (K words) Memory address Block 1 (K words) 01230123 Block N (K words) Word length 2 n - 1 Main memory addressed using n bits (total 2 n words) Cache memory has C rows

14. Cache memory work regime (cont.) Processor Cache memory Main memory Transfer of words Transfer of blocks

15. Reading from cache memory START Acquiring address from CPU Is this block’s address in the cache memory? EXECUTION Accessing main memory for the addressed block Assignment of the block to the cache memory row Transferring block into the cache memory Transferring word to CPU NO Transfer of word to CPU YES

16. Details of the cache memory Size Mapping Replacement algorithm Writing algorithm Row size Number of the cache memories

17. Size of the cache memory Minimization of the memory cost Maximization of the processor’s speed ProcessorType Prod. year L1 cache instruction L1 cache data L2 memory L3 memory IBM 360Mainframe196816-32 KBNone IBM 3033Mainframe197864 KBNone 80486PC19898 KBNone PentiumPC19938 KB 256/512None PowerPC G4 PC/serv.199932 KB 256/1 MB 2 MB Pentium 4PC/serv.20008 KB 256 KBNone ItaniumPC/serv.200116 KB 96 KB4 MB Athlon XpPC/serv.199964 KB 512 KBNone Athlon 64PC/serv.200264 KB 1 MBNone

18. Mapping function The number of the rows in the cache is smaller than the number of the blocks in the main memory Three methods exist: –Direct –Associative –Set-associative

19. Cache memory with direct mapping Comparison Flag RowWord Memory address Flag … Data L0L0 LiLi Main memory B0B0 W0 W1 W2 W3 s-rr w w s+w s w hit miss

20. Direct mapping (cont.) i – number of the row in the cache memory j – number of the block in the main memory m – number of rows in the cache memory i = j mod m Address length: s+w bits Number of the addressed units: 2 s+w words Block size = row size: 2 w words Number of blocks in the main memory: 2 s Number of rows in the cache memory: 2 r

21. Result of the direct mapping Row in the cache memory Assigned blocks in the main memory 00, m, 2m,..., 2 s – m 11, m+1, 2m+1,..., 2 s – m +1... m-1m-1, 2m-1, 3m-1,..., 2 s – 1

22. Example of the direct mapping For the cache memory having 2 14 rows (4 B each) and main memory of 16 MB capacity: Row in the cache memoryAssigned main memory blocks 0000000, 010000,..., FF0000 1000004, 010004,..., FF0004... 2 14 -100FFFC, 01FFFC,..., FFFFFC Row width: 8 b flag, 32 b data

23. Cache memory of associative mapping Comparison FlagWord Memory address Flag … Data L0L0 LiLi Main memory B0B0 W0 W1 W2 W3 s w w s+w s w hit miss s s

24. Associative mapping (cont.) Address length: s+w bits Number of the addressed units: 2 s+w words Block size = row size: 2 w words Number of the main memory blocks: 2 s Number of rows in the cache memory: any Flag size: s words

25. Example of the associative mapping Address 000000 000004 12357A FFFFF4 FFFFF8 FFFFFC Data 35281987 F235A72C 3982FB1A Flag Data 22 b 32 b Flag (22 b) Word (2 b) 000000 3FFFFF 048D5E 35281987 3982FB1A F235A72C

26. Cache memory with set-associative mapping Comparison Flag SectionWord Memory address Flag … Data S0S0 SiSi Main memory W0 W1 W2 W3 s-d d w s+w hit miss

27. Set-associative mapping (cont.) i – number of the row in the cache memory j – number of the block in the main memory m – number of rows in the cache memory m = v x k i = j mod v Address length: s+w bits Number of addressed units: 2 s+w words Block size = row size: 2 w words Number of blocks in the main memory: 2 s

28. Set-associative mapping (cont.) Number of rows in a section: k Number of sections: v = 2 d Number of rows in the cache memory: kv = k x 2 d Flag size: (s-d) bits

29. Example of the set-passociative mapping Flag 000 01A 1FF Data 35281987 F235A72C 67321342 3982FB1A Flag Data 9 b 32 b Flag (9 b) Słowo (2 b) 000 01A 35281987 67321342 Section (13 b) 0000 0004 7FFC 0000 0004 7FFC 9 b 32 b 01A F235A72C

30. Algorithms of the cache memory content replacement Least recently used (LRU) First in - first out (FIFO) Least frequently used (LFU) Random choice

31. Algorithms of writing into the cache memory write through write back System assuring consistency (multiprocessor system with cache) –Bus control with write through –Hardware transparency –Memory not mapped by the cache memory

32. Other problems Row size and block size Number of the cache memories –Memory of the higher level is integrated in one chip with the processor, works with identical frequency –Memory of the lower level works with the bus frequency (it is on the mainboard)

33. Pentium 4 cache memory

34. Pentium 4 processor core Instruction fetching/decoding unit –Fetches instructions from L2 cache memory –Decodes them into microoperations –transfers microoperations to L1 cache memory Non-sequential instruction execution unit –Queues microoperations Execution units –Execute microoperations –Fetch data from the L1 cache –Write results into the registers Memory subsystem –Communicates with the system bus and L2 cache memory

35. PowerPC cache memory ProcessorSizeB / rowOrganization PowerPC 6011 x 32 KB328-way PowerPC 6032 x 8 KB322-way PowerPC 6042 x 16 KB324-way PowerPC 6202 x 32 KB648-way PowerPC G32 x 32 KB648-way PowerPC G42 x 32 KB328-way

36. PowerPC cache memory (cont.)