Addressing Large Read Performance in Ext2fs

Slides:

Advertisements

Similar presentations

The Performance Impact of Kernel Prefetching on Buffer Cache Replacement Algorithms (ACM SIGMETRIC 05 ) ACM International Conference on Measurement & Modeling.

Advertisements

Part IV: Memory Management

Miss Penalty Reduction Techniques (Sec. 5.4) Multilevel Caches: A second level cache (L2) is added between the original Level-1 cache and main memory.

Discussion Week 7 TA: Kyle Dewey. Overview Midterm debriefing Virtual memory Virtual Filesystems / Disk I/O Project #3.

The Linux Kernel: Memory Management

Effects of Virtual Cache Aliasing on the Performance of the NetBSD Operating System Rafal Boni CS 535 Project Presentation.

Virtual Memory. The Limits of Physical Addressing CPU Memory A0-A31 D0-D31 “Physical addresses” of memory locations Data All programs share one address.

Paging Hardware With TLB

Virtual Memory Hardware Support

File Systems Examples.

Read vs. mmap Tan Li. Man mmap #include void *mmap(void *start, size_t length, int prot, int flags, int fd, off_t offset); int munmap(void *start, size_t.

Memory Management Design & Implementation Segmentation Chapter 4.

1 Introducing Collaboration to Single User Applications A Survey and Analysis of Recent Work by Brian Cornell For Collaborative Systems Fall 2006.

S.1 Review: The Memory Hierarchy Increasing distance from the processor in access time L1$ L2$ Main Memory Secondary Memory Processor (Relative) size of.

Computer ArchitectureFall 2007 © November 21, 2007 Karem A. Sakallah Lecture 23 Virtual Memory (2) CS : Computer Architecture.

Scheduler Activations Effective Kernel Support for the User-Level Management of Parallelism.

CS 300 – Lecture 22 Intro to Computer Architecture / Assembly Language Virtual Memory.

Memory Management 2010.

1  1998 Morgan Kaufmann Publishers Chapter Seven Large and Fast: Exploiting Memory Hierarchy (Part II)

ITEC 325 Lecture 29 Memory(6). Review P2 assigned Exam 2 next Friday Demand paging –Page faults –TLB intro.

1 Computer Performance: Metrics, Measurement, & Evaluation.

Silberschatz, Galvin and Gagne  2002 Modified for CSCI 346, Royden, Operating System Concepts Operating Systems Lecture 24 Paging.

Microprocessor-based systems Curse 7 Memory hierarchies.

8.4 paging Paging is a memory-management scheme that permits the physical address space of a process to be non-contiguous. The basic method for implementation.

Lecture Topics: 11/17 Page tables TLBs Virtual memory flat page tables

Background: Operating Systems Brad Karp UCL Computer Science CS GZ03 / M th November, 2008.

1 File Systems: Consistency Issues. 2 File Systems: Consistency Issues File systems maintains many data structures  Free list/bit vector  Directories.

Lecture 11 Page 1 CS 111 Online Memory Management: Paging and Virtual Memory CS 111 On-Line MS Program Operating Systems Peter Reiher.

Virtual Memory Chapter 8. Hardware and Control Structures Memory references are dynamically translated into physical addresses at run time –A process.

1 Memory Management Basics. 2 Program P Basic Memory Management Concepts Address spaces Physical address space — The address space supported by the hardware.

1 Memory Management. 2 Fixed Partitions Legend Free Space 0k 4k 16k 64k 128k Internal fragmentation (cannot be reallocated) Divide memory into n (possible.

Chapter 3.7 Segmentation. Memory allocation as a concept ● This presentation is about memory management specifically about memory segmentation and paging.

Project 4. “File System Implementation”

Lecture 4 Mechanisms & Kernel for NOSs. Mechanisms for Network Operating Systems  Network operating systems provide three basic mechanisms that support.

Linux file systems Name: Peijun Li Student ID: Prof. Morteza Anvari.

Consider the Java code snippet below. Is it a legal use of Java synchronization? What happens if two threads A and B call get() on an object supporting.

Virtual Memory Questions answered in this lecture: How to run process when not enough physical memory? When should a page be moved from disk to memory?

File System Performance CSE451 Andrew Whitaker. Ways to Improve Performance Access the disk less  Caching! Be smarter about accessing the disk  Turn.

Memory – Virtual Memory, Virtual Machines

Project 4. “File System Implementation”

Virtual memory.

Jonathan Walpole Computer Science Portland State University

CE 454 Computer Architecture

Chapter 2 Memory and process management

ECE232: Hardware Organization and Design

Protecting Memory What is there to protect in memory?

Protecting Memory What is there to protect in memory?

Memory Caches & TLB Virtual Memory

CS703 - Advanced Operating Systems

Outline Paging Swapping and demand paging Virtual memory.

143A: Principles of Operating Systems Lecture 6: Address translation (Paging) Anton Burtsev October, 2017.

Other Important Synchronization Primitives

Operating Systems (CS 340 D)

OpenAFS Linux Performance Improvements in 1. 5/1

Virtual Memory Chapter 8.

What we need to be able to count to tune programs

Lecture 14 Virtual Memory and the Alpha Memory Hierarchy

Making Virtual Memory Real: The Linux-x86-64 way

Page Replacement.

A Simulator to Study Virtual Memory Manager Behavior

Lecture 3: Main Memory.

Lecture 18 Syed Mansoor Sarwar

CSE451 Virtual Memory Paging Autumn 2002

Chapter 15: File System Internals

CSE 153 Design of Operating Systems Winter 19

Virtual Memory: Working Sets

File System Performance

Operating Systems: Internals and Design Principles, 6/E

Dirty COW Race Condition Attack

Presentation transcript:

Addressing Large Read Performance in Ext2fs - Analysis of 2.2 Kernel - Implementation of limited madvise system call. - Exploration into using madvise to improve readahead.

Motivations and Goals Understand the discrepancy between Win2k and Linux read performance. Evaluate 2.2 kernel code. Address perceived inadequacies of 2.2 kernel code Explore solutions outside of kernel code. Gain exposure to programming in the Linux kernel

Approach Use the mmap system call as a well defined entry point into the FS. Use “traditional” FS benchmarking to evaluate read performance. Bonnie and “mmapped” Bonnie Performance under “real” workloads such as an external sort (continuing work) printk, printk, printk….

Analysis of Linux Readahead Code inspection reveals a one-size-fits-all “readahead” mechanism. Number of readahead pages is fixed Readahead only done on page misses Can an application give the OS hints about how it will access its files? Introduce madvise() into 2.2 kernel Can we implement a “real” readahead mechanism?

Implementing madvise() Advice to the kernel’s readahead mechanism. Specifically: “behavior” flag Advice “level” As well as virtual address and length parameters. int madvise(addr_t addr_p, ulong bytes, uint behavior, uint level);

How Effective Is Our Advice?

How Advice Affects Run Time

How Advice Affects Run Time

Extreme Advice

hdparm to the Rescue?

hdparm to the Rescue?

Attempting “true” readahead Why wait for page cache misses to initiate readahead? Instead, periodically initiate readahead on page cache hits. How often should we initiate readahead? Which pages should we readahead? How many? Don’t block on readaheads! Nice to have, but not worth the wait

Implementation Details Divide file into groups of 2advice pages When user program is accessing pages in group i, initiate readahead on pages in group (i+1) Period: initiate readahead every 16 page accesses (roughly) Will likely initiate readahead on group (I+1) multiple times—a feature. The perfect solution? Well, no…but…

“True” readahead Performance

“True” readahead Performance

Head to Head Comparison

What Next? Further tweaking of readaheads: Proof of principle with advice level 6, but let’s improve other advice levels as well To what extent can we eliminate idle time? Test under “real world” workloads: Eg. External Sort (already written) Further excursions into the kernel code to better understand “requests” But avoid getting lost in assembly code