C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Advanced Computers Architecture Virtual Memory By Rohit Khokher Department of Computer Science, Sharda University, Greater Noida, India

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Topic s of my Talk 1. Principles of Virtual Memory 2. Implementations of Virtual Memory – Paging – Segmentation – Paging With Segmentation – Paging of System Tables – Translation Look-aside Buffers 3. Memory Allocation in Paged Systems – Global Page Replacement Algorithms – Local Page Replacement Algorithms – Load Control and Thrashing – Evaluation of Paging

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Hardware and Control Structures Memory references are dynamically translated into physical addresses at run time – A process may be swapped in and out of main memory such that it occupies different regions A process may be broken up into pieces that do not need to located contiguously in main memory All pieces of a process do not need to be loaded in main memory during execution

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Execution of a Program Operating system brings into main memory a few pieces of the program Resident set - portion of process that is in main memory An interrupt is generated when an address is needed that is not in main memory Operating system places the process in a blocking state

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Piece of process that contains the logical address is brought into main memory – Operating system issues a disk I/O Read request – Another process is dispatched to run while the disk I/O takes place – An interrupt is issued when disk I/O complete which causes the operating system to place the affected process in the Ready state

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Advantages of Breaking up a Process More processes may be maintained in main memory – Only load in some of the pieces of each process – With so many processes in main memory, it is very likely a process will be in the Ready state at any particular time A process may be larger than all of main memory

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Types of Memory Real memory – Main memory Virtual memory – Memory on disk – Allows for effective multiprogramming and relieves the user of tight constraints of main memory

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Virtual Memory Virtual memory is a technique that allows processes that may not be entirely in the memory to execute by means of automatic storage allocation upon request.automatic storage allocation The term virtual memory refers to the abstraction of separating LOGICAL memory-- memory as seen by the process--from PHYSICAL memory--memory as seen by the processor. Because of this separation, the programmer needs to be aware of only the logical memory space while the operating system maintains two or more levels of physical memory space. The virtual memory abstraction is implemented by using secondary storage to augment the processor's main memory. Data is transferred from secondary to main storage as and when necessary and the data replaced is written back to the secondary storage according to a predetermined replacement algorithm.replacement algorithm If the data swapped is designated a fixed size, this swapping is called paging; if variable sizes are permitted and the data is split along logical lines such as subroutines or matrices, it is called segmentation. Some operating systems combine segmentation and paging.segmentation

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 The diagram illustrates that a program generated address ( 1 ) or "logical address" consisting of a logical page number plus the location within that page (x) must be interpreted or "mapped" onto an actual (physical) main memory address by the operating system using an address translation function or mapper mapper ( 2 ). If the page is present in the main memory, the mapper substitutes the physical page frame number for the logical number ( 3 ). If the mapper detects that the page requested is not present in main memory, a fault occurs and the page must be read into a frame in main memory from secondary storage

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 What do you think are the main considerations in implementing this virtual memory system? Assignment

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 A System with Physical Memory Only CPU Memory Examples: – most Cray machines, early PCs, nearly all embedded systems, etc. Addresses generated by the CPU point directly to bytes in physical memory Physical Addresses 0: 1: N-1:

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 A System with Virtual Memory 12 Examples: – workstations, servers, modern PCs, etc. Address Translation: Hardware converts virtual addresses to physical addresses via an OS-managed lookup table (page table) CPU 0: 1: N-1: Memory 0: 1: P-1: Page Table Disk Virtual Addresses Physical Addresses

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Page Faults (Similar to “Cache Misses”) 13 CPU Memory Page Table Disk Virtual Addresses Physical Addresses CPU Memory Page Table Disk Virtual Addresses Physical Addresses Before fault After fault What if an object is on disk rather than in memory? – Page table entry indicates virtual address not in memory – OS exception handler invoked to move data from disk into memory current process suspends, others can resume OS has full control over placement, etc.

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Mapper The mapper is the part of the operating system that translates the logical page number generated by the program into the physical page frame number where the main memory holds the page.logical page number This translation is accomplished by using a directly indexed table called the page table which identifies the location of all the program's pages in the main store. page table If the page table reveals that the page is, in fact, not resident in the main memory, the mapper issues a page fault to the operating system so that execution is suspended on the process until the desired page can be read in from the secondary store and placed in main memory.

15 Virtual memory (VM) is – Not a physical device but an abstract concept – Comprised of the virtual address spaces (of all processes) Virtual address space (VAS) (of one process) – Set of visible virtual addresses – (Some systems may use a single VAS for all processes) Resident set – Pieces of a process currently in physical memory Working set – Set of pieces a process is currently working on Terms & Notions Advanced Computers Architecture, UNIT 2

16 Principle of Locality: – Memory references within a process tend to cluster – Working set should be part of the resident set to operate efficiently (else: frequent memory access faults)  honor the principle of locality to achieve this repeated references: single jumps: working set: initialization data initialization code early phase of process lifetime code 1code 2data main phase of process lifetime finalization code final phase of process lifetime The Principle of Locality Advanced Computers Architecture, UNIT 2

17 Danger: Thrashing: – „Piece“ just swapped out is immediately requested again – System swaps in/out all the time, no real work is done Thus: „piece“ for swap out has to be chosen carefully – Keep track of „piece“ usage („age of piece“) – Hopefully „piece“ used frequently lately will be used again in near future (principle of locality!) lack of memory find rarely used „piece" adjust mapping table „piece“ modified? save HDD location of „piece“ discard „piece“ no write „piece“ out to disk yes no need to swap out complete process!!! VM: Features Swapping Advanced Computers Architecture, UNIT 2

18 Each process has its own virtual address space – Processes invisible to each other – Process cannot access another processes memory MMU checks protection bits on memory access (during address mapping) – „Pieces“ can be protected from being written to or being executed or even being read – System can distinguish different protection levels (user / kernel mode) Write protection can be used to implement copy on write (  Sharing) VM: Features Protection Advanced Computers Architecture, UNIT 2

19 Pieces of different processes mapped to one single „piece“ of physical memory – Allows sharing of code (saves memory), e.g. libraries – Copy on write: „piece“ may be used by several processes until one writes to it (then that process gets its own copy) – Simplifies interprocess-communication (IPC) Piece 2 Piece 1 Virtual memory Process 1 Piece 0 Piece 1 Piece 2 Piece 0 Physical memory shared memory Virtual memory Process 2 Piece 1 Piece 0 Piece 2 VM: Features Sharing Advanced Computers Architecture, UNIT 2

20 VM supports – Swapping Rarely used „pieces“ can be discarded or swapped out „Piece“ can be swapped back in to any free piece of physical memory large enough, mapping unit translates addresses – Protection – Sharing Common data or code may be shared to save memory Process need not be in memory as a whole – No need for complicated overlay techniques (OS does job) – Process may even be larger than all of physical memory – Data / code can be read from disk as needed VM: Advantages (1) Advanced Computers Architecture, UNIT 2

21 Code can be placed anywhere in physical memory without relocation (adresses are mapped!) Increased cpu utilization – more processes can be held in memory (in part)  more processes in ready state (consider: 80% HDD I/O wait time not uncommon) VM: Advantages (2) Advanced Computers Architecture, UNIT 2

22 Memory requirements (mapping tables) Longer memory access times (mapping table lookup) – Can be improved using TLB VM: Disadvantages Advanced Computers Architecture, UNIT 2

23 VM may be implemented using – Paging – Segmentation – Combination of both VM: Implementation Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Address Translation Address translation must be fast (it happens on every memory access). We need a fully associative placement policy. miss penalty is huge! We can’t afford to go looking at every virtual page to find the right one we don’t use the tag bits approach

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 VM Address Translation V = {0, 1,..., N–1} virtual address space P = {0, 1,..., M–1} physical address space MAP: V  P U {  } address mapping function N > M MAP(a) = a' if data at virtual address a is present at physical address a' in P =  if data at virtual address a is not present in P Processor Hardware Addr Trans Mechanism fault handler Main Memory Secondary memory a a'  page fault physical address OS performs this transfer (only if miss) virtual addresspart of the on-chip memory mgmt unit (MMU)

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 VM Address Translation Parameters – P = 2 p = page size (bytes). – N = 2 n = Virtual address limit – M = 2 m = Physical address limit virtual page numberpage offset virtual address physical page numberpage offset physical address 0p–1 address translation pm–1 n–10p–1p Notice that the page offset bits don't change as a result of translation

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Page Tables Virtual Page Number Memory resident page table (physical page or disk address) Physical Memory Disk Storage (swap file or regular file system file) Valid

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Address Translation via Page Table virtual page number (VPN)page offset virtual address physical page number (PPN)page offset physical address 0p–1pm–1 n–10p–1p page table base register if valid=0 then page not in memory valid physical page number (PPN) access VPN acts as table index

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Page Table Operation Translation – Separate (set of) page table(s) per process – VPN forms index into page table (points to a page table entry) Computing Physical Address – Page Table Entry (PTE) provides information about page if (valid bit = 1) then the page is in memory. – Use physical page number (PPN) to construct address if (valid bit = 0) then the page is on disk – Page fault – Must load page from disk into main memory before continuing Checking Protection – Access rights field indicate allowable access e.g., read-only, read-write, execute-only typically support multiple protection modes (e.g., kernel vs. user) – Protection violation fault if user doesn’t have necessary permission

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Integrating VM and Cache CPU Trans- lation Cache Main Memory VAPA miss hit data Most Caches “Physically Addressed” – Accessed by physical addresses – Allows multiple processes to have blocks in cache at same time – Allows multiple processes to share pages – Cache doesn’t need to be concerned with protection issues Access rights checked as part of address translation Perform Address Translation Before Cache Lookup – But this could involve a memory access itself (of the PTE) – Of course, page table entries can also become cached

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Speeding up Translation with a TLB “ Translation Lookaside Buffer” (TLB) – Small hardware cache in MMU – Maps virtual page numbers to physical page numbers – Contains complete page table entries for small number of pages CPU TLB Lookup Cache Main Memory VAPA miss hit data Trans- lation hit miss

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Address Translation with a TLB virtual address virtual page number page offset physical address n–10p–1p validphysical page numbertag validtagdata = cache hit tagbyte offset index = TLB hit TLB Cache...

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Simple Memory System Example Addressing – 14-bit virtual addresses – 12-bit physical address – Page size = 64 bytes VPO PPOPPN VPN (Virtual Page Number) (Virtual Page Offset) (Physical Page Number) (Physical Page Offset)

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2 Simple Memory System Page Table – Only show first 16 entries VPNPPNValidVPNPPNValid – A B–0 04–00C– D2D1 06–00E111 07–00F0D1

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2

C SINGH, JUNE 7-8, 2010IWW 2010, ISATANBUL, TURKEY Advanced Computers Architecture, UNIT 2