read vs. mmap Tan Li

Man mmap #include void *mmap(void *start, size_t length, int prot, int flags, int fd, off_t offset); int munmap(void *start, size_t length); The mmap method's arguments include an address, start, total size to be mapped length, file protection prot, mapping flags, a file descriptor fd for an open file and an offset. The flags allow different options to be passed to mmap to control the way a file is mapped to the process address space. buf = mmap(0, infilestat.st_size, PROT_READ, MAP_SHARED, infile,0)

Usage of mmap The mmap() function allows access to resources via address space manipulations, instead of read()/ write(). Once a file is mapped, all a process has to do to access it is use the data at the address to which the file was mapped. So, using pseudo- code to illustrate the way in which an existing program might be changed to use mmap(), the following: fildes = open(...) lseek(fildes, some_offset) read(fildes, buf, len) /* Use data in buf. */ becomes: fildes = open(...) address = mmap(0, len, PROT_READ, MAP_PRIVATE, fildes, some_offset) /* Use data at address. */

Advantages In traditional file I/O involving read, data is copied from the disk to a kernel buffer, then the kernel buffer is copied into the process's heap space for use. In memory-mapped file I/O, data is copied from the disk straight into the process's address space, into the segment where the file is mapped. The file is then accessed by references to memory locations with pointers. This way, I/O on the file is done without the overhead of handling the data twice.

Disadvantages 1. overheads related the algorithms the operating system *MUST* use to figure out which pages to remove (on the fly) when the data set does not fit in main memory, and there are overheads related to the heuristics the operating system employs to try to predict the memory usage pattern to perform some read-ahead. 2. on a 32-bit system, you're certainly not going to be able to mmap() any file larger than 4 GB. The real limit may be a lot lower (1 GB or less), since parts of the address space may be reserved for the kernel, or already used by the program. It might also be fragmented. For these reasons, you usually want to have code to fall back to read() in case the mmap fails. This would be good for portability as well. 3. Use of mmap() may reduce the amount of memory available to other memory allocation functions.

Simulation Result

Usage of madvise() The madvise() system call advises the kernel about how to handle paging input/output in the address range beginning at address start and with size length bytes. It allows an application to tell the kernel how it expects to use some mapped or shared memory areas, so that the kernel can choose appropriate read-ahead and caching techniques. This call does not influence the semantics of the application (except in the case of MADV_DONTNEED), but may influence its performance. The kernel is free to ignore the advice. MADV_SEQUENTIAL Expect page references in sequential order. (Hence, pages in the given range can be aggressively read ahead, and may be freed soon after they are accessed.)

Suggestion madvise() + mmap() with backup meachanics of fread() read() with O_DIRECT with backup meachanics of fread().