1. Computer Architecture
Part III-B: Cache Memory

2. Access Time
If every memory reference to cache required transfer of one word between MM and cache, no increase in speed is achieved. In fact, speed will drop because apart from MM access, there is additional access to cache
Suppose reference is repeated n times, and after the first reference, location is always found in the cache
Average access time:
tc= cache access time
tm = main memory access time
n = number of accesses/references

3. Cache Hit Ratio The probability that a word will be found in the cache
Depends upon the program and the size and organization of the cache
h = Number of times required word found in cache
Total number of references
h: hit ratio

4. Access Time ta = Average access time tc = Cache access time
(1-h) = miss ratio
tm = Memory access time

5. Fetch Mechanisms Demand Fetch Prefetch Selective Fetch
Fetch a block from memory when it is needed and is not in the cache
Prefetch
Fetch block/s from memory before they are requested
Selective Fetch
Not always fetching blocks, dependent on some defined criterion; blocks are stored in MM rather than the cache

6. Write Mechanisms
When words are read from the cache, contents are not modified
However, in general, cache data can be modified and it is possible that data in cache is different from data in MM
Two mechanisms to keep cache and MM in sync
Write-through mechanism
Write-back mechanism

7. Synchronization Mechanisms
Write-through
Every write operation to the cache is simultaneously repeated for MM
Write-back
The write operation to MM is only done during block-replacement time (i.e. a block displaced by incoming block might be written back to MM, regardless whether the block was altered or not)

8. Replacement Algorithms
When the word being requested by the CPU is not in the cache, it needs to be transferred from MM. (or it can also be from secondary memory to MM)
A page fault occurs when a page or a block is not in the cache (or MM in the case of secondary memory)
Replacement algorithms determine which page/block to remove or overwrite

9. Characteristics Usage based or Non-usage based
Usage based : the choice of page/block to replace is dependent on the how many times each page/block has been referenced
Non-usage based : Use some other criteria for replacement

10. Assumptions
For a given page size, we only need to consider the page/block number.
If we have a reference (hit) to a page p, then any immediately succeeding references to p does not cause a page fault
The size of memory/cache is represented as the number of pages it is capable of holding (page frame )

11. Example Consider the following address sequence calls:
which, at 100 bytes per page, can be reduced to the following access string:
This sequence of page requests is called a reference string.

12. Replacement Policies Random replacement algorithm
First-in first-out replacement
Optimal Algorithm
Least recently used algorithm
Least Frequently Used
Most Frequently Used

13. Random Replacement A page is chosen randomly at page fault time
There is no relationship between the pages or their use.
Choice is done by a random number generator.

14. FIFO Memory treated as a queue Easy to understand and program
When a page comes in, it is inserted at the tail
When a page is removed, the entry at the head of the queue gets deleted
Easy to understand and program
Performance is not consistently good; dependent on reference string

15. FIFO Example Consider the following reference string:
With a page frame of 3
* * * * * * * *
An * indicates a miss (the page requested by the CPU
is not in the cache or in MM)

16. FIFO Example #2 Consider the following reference string:
With a page frame of 3
* * * * * * * * *
We have 9 page faults
Try performing this FIFO with a page frame of 4

17. Belady’s Anomaly
An increase in page frame does not necessarily mean a decrease in page faults
More formally, Belady’s anomaly reflects the fact that, for some page-replacement algorithms, the page fault rate may increase as the number of allocated frames increases

18. Optimal Algorithm
The page that will not be used for the longest period of time is replaced
Guarantees the lowest page fault rate for a fixed number of frames
Difficult to implement because it requires future knowledge of the reference string

19. Optimal Algorithm Example
Consider the following reference string:
With a page frame of 3
We look ahead and see that 7 is the page which will not be used again, so we replace 7; we also note that after our first hit we should not replace 0 immediately, but rather 1 because 1 will not be referenced any more (2 will be referenced last.)
* * * * * *

20. Least Recently Used Approximates the optimal algorithm
Replaces the page that has not been used for the longest period of time
When all page frames have been used up and every time there is a page hit, the referenced page is placed at the tail to indicate it has been recently accessed

21. LRU Example Consider the following reference string:
With a page frame of 3
* * * * * * *
We have 7 page faults
Try performing this LRU with a page frame of 4

22. Least Frequently Used
Counts the number of references made to each page; when page is accessed, counter is incremented by one
Page with smallest count is replaced
FIFO is used to resolve a tie
Rationale: Page with the bigger counter is an actively used page
Problem
Page initially actively may never be used again
Solved by using a decaying counter

23. LFU Example Consider the following reference string:
With a page frame of 3
* * * * * * *
We have 7 page faults
Try performing this LFU with a page frame of 4

24. Most Frequently Used Opposite of LFU
Replace page with the highest count
Tie is resolved using FIFO
Based on the argument that the page with smallest count has just been probably brought in and is yet to be used
Both LFU and MFU are not common and implementation is expensive.