Win32 Programming Lesson 18: More Memory Mapped Files and the HEAP (Finally, cool stuff!)

Where are we?  We’ve gotten pretty familiar with the idea of memory-mapped files but there are some important concepts we haven’t looked at  Finish up today, and then look at the heap

Sharing Data  There are lots of different ways to share data between processes  But the “lowest level” way is really in memory, via a memory-mapped file Two or more processes map the same base address – hence they are sharing the same physical memory

Avoiding the “real” File system  Very inconvenient if every memory-mapped file actually had to exist on disk Imagine that you simply wanted to use the mechanism to pass data, not keep it Can call CreateFileMapping with INVALID_HANDLE_VALUE as the hFile parameter, and the memory-mapped file is backed by the page file

30 Second Quiz  What’s wrong with this code?  HANDLE hFile = CreateFile(...); HANDLE hMap = CreateFileMapping(hFile,...); if (hMap == NULL) return(GetLastError());

Answer…  You’ll get back INVALID_HANDLE_VALUE from the first call  Which means…

MMFExample  Simple program  When it maps the view of the file, it transfers data between the two programs  Nice method of sharing between two processes!

Sparsely Committing…  Remember we talked about how to commit memory for files?  Same discussion for Memory-mapped files – that is, we don’t need to commit all our memory at once

Consider  CELLDATA CellData[200][256];  If sizeof(CELLDATA) is 128 that’s about 6MB.  Better to share as a sparsely-committed file mapping object  If we’re sharing via the paging file, can use SEC_RESERVE or SEC_COMMIT

SEC_RESERVE  When you pass in SEC_RESERVE you don’t actually commit the space  Just returns a HANDLE to the file mapping object  Any attempts to access the memory cause a memory violation

VirtualAlloc (again)  Solution: Call VirtualAlloc to allocate the memory as we use it!

The HEAP  Heap is a fantastic tool for allocating small blocks of memory Perfect for linked lists and trees Advantage: can ignore allocation granularity Disadvantage: slow, with no direct control of physical allocation Better yet, internals not entirely documented

Default  Each process gets a default heap of 1MB  Can specify at link time (/HEAP:)  Used by many Windows/C RTL functions  Access to the HEAP is serialized (why, and what does this mean?)  Can obtain a handle via: HANDLE GetProcessHeap();

More than 1 Heap is a…  Five reasons you might want to do this: Component protection More efficient memory management Local access Avoiding thread sync overhead Quick free

1: Component Protection  Imagine you have two structures: a linked list and a binary tree  If you share one heap, and one has a bug, the problem may show up in the other structure  If we have different heaps, problems tend to be localized (unless you *really* mess up!)

2: Memory Management  Heaps work best when all the objects in them are the same size  Imagine mixing two different sizes; when you free object 1 object 2 may not be a perfect fit  Better to allocate all objects of the same size in the same place

3: Local Access  Swapping to disk is really expensive  Best to keep things you use together close together

4: Avoiding thread-sync issues  Heaps are serialized by default  CPU overhead involved in keeping heap access thread safe  If you tell the system a heap is single- threaded, you can lose this overhead DANGER WILL ROBINSON: YOU ARE NOW RESPONSIBLE FOR THREAD SAFETY!!!

5: Quick free  Instead of freeing things up block by block you can choose to free the entire heap in one go  That’s *really* quick

So… how?  HANDLE HeapCreate( DWORD fdwOptions, SIZE_T dwInitialSize, SIZE_T dwMaximumSize);  Options: 0, HEAP_NO_SERIALIZE, HEAP_GENERATE_EXCEPTIONS  Serialize turns off checking for Alloc and Free  Can manage this yourself via Critical Sections if you want to

HEAP_GENERATE_EXCEPTIONS  Raise an exception whenever an allocation request fails  Basically, it’s just about whether you want to catch exceptions or check return values – depends on the application  Oh… if dwMaximumSize is 0 the size is unlimited…

Allocating Memory from the Heap  PVOID HeapAlloc( HANDLE hHeap, DWORD fdwFlags, SIZE_T dwBytes);  Flags: HEAP_ZERO_MEMORY, HEAP_GENERATE_EXCEPTIONS, HEAP_NO_SERIALIZE  Exceptions: STATUS_NO_MEMORY, STATUS_ACCESS_VIOLATION

Changing the size…  PVOID HeapReAlloc( HANDLE hHeap, DWORD fdwFlags, PVOID pvMem, SIZE_T dwBytes);  New flag: HEAP_REALLOC_IN_PLACE_ONLY  Means that the location won’t change

Obtaining the Size  SIZE_T HeapSize( HANDLE hHeap, DWORD fdwFlags, LPCVOID pvMem);  Flags: 0 or HEAP_NO_SERIALIZE

Freeing a block  BOOL HeapFree( HANDLE hHeap, DWORD fdwFlags, PVOID pvMem);  Flags? You tell me…

Destroying a Heap  BOOL HeapDestroy(HANDLE hHeap);  TRUE on success  You can’t destroy the default heap!

Heaps with C++  Under C you would use malloc  In C++ can use new/delete CSomeClass *pSomeClass = new CSomeClass; delete pSomeClass; Now the clever bit: overload new/delete…

Prototype  class CSomeClass { private: static HANDLE s_hHeap; static UINT s_uNumAllocsInHeap; // Other private data and member functions public: void* operator new (size_t size); void operator delete (void* p); // Other public data and member functions };

Code… HANDLE CSomeClass::s_hHeap = NULL; UINT CSomeClass::s_uNumAllocsInHeap = 0; void* CSomeClass::operator new (size_t size) { if (s_hHeap == NULL) { // Heap does not exist; create it. s_hHeap = HeapCreate(HEAP_NO_SERIALIZE, 0, 0); if (s_hHeap == NULL) return(NULL); } // The heap exists for CSomeClass objects. void* p = HeapAlloc(s_hHeap, 0, size); if (p != NULL) { // Memory was allocated successfully; increment // the count of CSomeClass objects in the heap. s_uNumAllocsInHeap++; } // Return the address of the allocated CSomeClass object. return(p); }

And delete… void CSomeClass::operator delete (void* p) { if (HeapFree(s_hHeap, 0, p)) { // Object was deleted successfully. s_uNumAllocsInHeap--; } if (s_uNumAllocsInHeap == 0) { // If there are no more objects in the heap, // destroy the heap. if (HeapDestroy(s_hHeap)) { // Set the heap handle to NULL // so that the new operator // will know to create a new heap if a // new CSomeClass // object is created. s_hHeap = NULL; } } }

Misc Functions…  DWORD GetProcessHeaps – returns handles to all heaps in the process  BOOL HeapValidate – check that a heap is a-okay…  UINT HeapCompact – coalesce free blocks  HeapLock and HeapUnlock – used for thread sync  HeapWalk – for debugging; lets you enumerate sections/blocks in the heap

Next  Next, it gets difficult DLLs