Background: I/O Concurrency Brad Karp UCL Computer Science CS GZ03 / M030 2 nd October, 2008.

Slides:



Advertisements
Similar presentations
Threads, SMP, and Microkernels
Advertisements

More on Processes Chapter 3. Process image _the physical representation of a process in the OS _an address space consisting of code, data and stack segments.
Threads, Events, and Scheduling Andy Wang COP 5611 Advanced Operating Systems.
Threads, SMP, and Microkernels Chapter 4. Process Resource ownership - process is allocated a virtual address space to hold the process image Scheduling/execution-
Chapter 4 Threads, SMP, and Microkernels Patricia Roy Manatee Community College, Venice, FL ©2008, Prentice Hall Operating Systems: Internals and Design.
Computer Systems/Operating Systems - Class 8
Chapter 7 Protocol Software On A Conventional Processor.
ECE 526 – Network Processing Systems Design Software-based Protocol Processing Chapter 7: D. E. Comer.
1 Threads, SMP, and Microkernels Chapter 4. 2 Process: Some Info. Motivation for threads! Two fundamental aspects of a “process”: Resource ownership Scheduling.
1 Processes and Pipes COS 217 Professor Jennifer Rexford.
Precept 3 COS 461. Concurrency is Useful Multi Processor/Core Multiple Inputs Don’t wait on slow devices.
Computer Science 162 Discussion Section Week 2. Agenda Recap “What is an OS?” and Why? Process vs. Thread “THE” System.
Threads vs. Processes April 7, 2000 Instructor: Gary Kimura Slides courtesy of Hank Levy.
Threads 1 CS502 Spring 2006 Threads CS-502 Spring 2006.
Operating Systems Concepts Professor Rick Han Department of Computer Science University of Colorado at Boulder.
1 I/O Management in Representative Operating Systems.
Threads CS 416: Operating Systems Design, Spring 2001 Department of Computer Science Rutgers University
1 Threads Chapter 4 Reading: 4.1,4.4, Process Characteristics l Unit of resource ownership - process is allocated: n a virtual address space to.
Threads math 442 es Jim Fix. Reality vs. Abstraction A computer’s OS manages a lot: multiple users many devices; hardware interrupts multiple application.
ThreadsThreads operating systems. ThreadsThreads A Thread, or thread of execution, is the sequence of instructions being executed. A process may have.
Scheduler Activations Jeff Chase. Threads in a Process Threads are useful at user-level – Parallelism, hide I/O latency, interactivity Option A (early.
Basics of Operating Systems March 4, 2001 Adapted from Operating Systems Lecture Notes, Copyright 1997 Martin C. Rinard.
Processes Part I Processes & Threads* *Referred to slides by Dr. Sanjeev Setia at George Mason University Chapter 3.
CSE 451: Operating Systems Autumn 2013 Module 6 Review of Processes, Kernel Threads, User-Level Threads Ed Lazowska 570 Allen.
1 CS503: Operating Systems Part 1: OS Interface Dongyan Xu Department of Computer Science Purdue University.
Chapter 4 Threads, SMP, and Microkernels Dave Bremer Otago Polytechnic, N.Z. ©2008, Prentice Hall Operating Systems: Internals and Design Principles, 6/E.
1 Chapter Client-Server Interaction. 2 Functionality  Transport layer and layers below  Basic communication  Reliability  Application layer.
Operating System 4 THREADS, SMP AND MICROKERNELS
Threads, Thread management & Resource Management.
Implementing Processes and Process Management Brian Bershad.
Chapter 4 Threads, SMP, and Microkernels Patricia Roy Manatee Community College, Venice, FL ©2008, Prentice Hall Operating Systems: Internals and Design.
Operating Systems Lecture 2 Processes and Threads Adapted from Operating Systems Lecture Notes, Copyright 1997 Martin C. Rinard. Zhiqing Liu School of.
Processes and Threads Processes have two characteristics: – Resource ownership - process includes a virtual address space to hold the process image – Scheduling/execution.
Lecture 3 Process Concepts. What is a Process? A process is the dynamic execution context of an executing program. Several processes may run concurrently,
Operating Systems Lecture 7 OS Potpourri Adapted from Operating Systems Lecture Notes, Copyright 1997 Martin C. Rinard. Zhiqing Liu School of Software.
Threads, SMP, and Microkernels Chapter 4. Process Resource ownership - process is allocated a virtual address space to hold the process image Scheduling/execution-
Background: Operating Systems Brad Karp UCL Computer Science CS GZ03 / M th November, 2008.
Scheduling Lecture 6. What is Scheduling? An O/S often has many pending tasks. –Threads, async callbacks, device input. The order may matter. –Policy,
1 Threads, SMP, and Microkernels Chapter 4. 2 Process Resource ownership: process includes a virtual address space to hold the process image (fig 3.16)
1 Threads, SMP, and Microkernels Chapter Multithreading Operating system supports multiple threads of execution within a single process MS-DOS.
CSE 451: Operating Systems Winter 2015 Module 5 1 / 2 User-Level Threads & Scheduler Activations Mark Zbikowski 476 Allen Center.
Lecture 5: Threads process as a unit of scheduling and a unit of resource allocation processes vs. threads what to program with threads why use threads.
1 Computer Systems II Introduction to Processes. 2 First Two Major Computer System Evolution Steps Led to the idea of multiprogramming (multiple concurrent.
Operating Systems CSE 411 CPU Management Sept Lecture 10 Instructor: Bhuvan Urgaonkar.
Concurrency, Processes, and System calls Benefits and issues of concurrency The basic concept of process System calls.
Department of Computer Science and Software Engineering
Processes, Threads, and Process States. Programs and Processes  Program: an executable file (before/after compilation)  Process: an instance of a program.
Processes and Virtual Memory
The Mach System Silberschatz et al Presented By Anjana Venkat.
Introduction Contain two or more CPU share common memory and peripherals. Provide greater system throughput. Multiple processor executing simultaneous.
Operating System Concepts
Threads, SMP, and Microkernels Chapter 4. Processes and Threads Operating systems use processes for two purposes - Resource allocation and resource ownership.
CSCI/CMPE 4334 Operating Systems Review: Exam 1 1.
Threads prepared and instructed by Shmuel Wimer Eng. Faculty, Bar-Ilan University 1July 2016Processes.
Process Management Process Concept Why only the global variables?
CS 6560: Operating Systems Design
Chapter 4: Multithreaded Programming
CS703 - Advanced Operating Systems
CSE 451: Operating Systems Spring 2012 Module 6 Review of Processes, Kernel Threads, User-Level Threads Ed Lazowska 570 Allen.
Lecture Topics: 11/1 General Operating System Concepts Processes
CSE 451: Operating Systems Autumn 2003 Lecture 5 Threads
Threads Chapter 4.
CSE 451: Operating Systems Winter 2003 Lecture 5 Threads
Prof. Leonardo Mostarda University of Camerino
Chapter 13: I/O Systems I/O Hardware Application I/O Interface
Threads vs. Processes Hank Levy 1.
CS510 Operating System Foundations
CSE 153 Design of Operating Systems Winter 2019
CS703 – Advanced Operating Systems
CSE 451: Operating Systems Winter 2001 Lecture 5 Threads
Presentation transcript:

Background: I/O Concurrency Brad Karp UCL Computer Science CS GZ03 / M030 2 nd October, 2008

2 Outline “Worse Is Better” and Distributed Systems Problem: Naïve single-process server leaves system resources idle; I/O blocks –Goal: I/O concurrency –Goal: CPU concurrency Solutions –Multiple processes –One process, many threads –Event-driven I/O (not in today’s lecture)

3 Review: How Do Servers Use Syscalls? Consider server_1() web server (in handout) time application CPU disk syscalls network syscalls Server waits for each resource in turn Each resource largely idle What if there are many clients? R RWW R C

4 Performance and Concurrency Under heavy load, server_1(): –Leaves resources idle –…and has a lot of work to do! Why? –Software poorly structured! –What would a better structure look like?

5 Solution: I/O Concurrency Can we overlap I/O with other useful work? Yes: –Web server: if files in disk cache, I/O wait spent mostly blocked on write to network –Networked file system client: could compile first part of file while fetching second part Performance benefits potentially huge –Say one client causes disk I/O, 10 ms –If other clients’ requests in cache, could serve 100 other clients during that time!

6 One Process May Be Better Than You Think OS provides I/O concurrency to application transparently when it can, e.g., –Filesystem does read-ahead into disk buffer cache; write-behind from disk buffer cache –Networking code copies arriving packets into application’s kernel socket buffer; copies app’s data into kernel socket buffer on write()

7 I/O Concurrency with Multiple Processes Idea: start new UNIX process for each client connection/request Master process assigns new connections to child processes Now plenty of work to keep system busy! –One process blocks in syscall, others can process arriving requests Structure of software still simple –See server_2() in webserver.c –fork() after accept() –Otherwise, software structure unchanged!

8 Multiple Processes: More Benefits Isolation –Bug while processing one client’s request leaves other clients/requests unaffected –Processes do interact, but OS arbitrates (e.g., “lock the disk request queue”) CPU concurrency for “free” –If more than one CPU in box, each process may run on one CPU

9 CPU Concurrency Single machine may have multiple CPUs, one shared memory –Symmetric Multiprocessor (SMP) PCs –Intel Core Duo I/O concurrency tools often help with CPU concurrency –But way more work for OS designer! Generally, CPU concurrency way less important than I/O concurrency –Factor of 2X, not 100X –Very hard to program to get good scaling –Easier to buy 2 machines (see future lectures!)

10 Problems with Multiple Processes fork() may be expensive –Memory for new address space –300 us minimum on modern PC running UNIX Processes fairly isolated by default –Memory not shared –How do you build web cache on server visible to all processes? –How do you simply keep statistics?

11 Concurrency with Threads Similar to multiple processes Difference: one address space –All threads share same process’ memory –One stack per thread, inside process Seems simple: single-process structure! Programmer needs to use locks One thread can corrupt another (i.e., no cross-request isolation)

12 Concurrency with Threads Kernel User Space Filesystem Disk Driver Hardware App1App2 00 N M t1 stack t2 stack

13 Threads: Low-Level Details Are Hard! Suppose thread calls read() (or other blocking syscall) –Does whole process block until I/O done? –If so, no I/O concurrency! Two solutions: –Kernel-supported threads –User-supported threads

14 Kernel-Supported Threads OS kernel aware of each thread –Knows if thread blocks, e.g., disk read wait –Can schedule another thread Kernel requirements: –Per-thread kernel stack –Per-thread tables (e.g., saved registers) Semantics: –Per-process: address space, file descriptors –Per-thread: user stack, kernel stack, kernel state

15 Kernel-Supported Threads Kernel User Space Filesystem Disk Driver Hardware App1App2 00 N M t1 stack stack, table t2 stack stack, table

16 Kernel Threads: Trade-Offs Kernel can schedule one thread per CPU –Fits our goals well: both CPU and I/O concurrency But kernel threads expensive, like processes: –Kernel must help create each thread –Kernel must help with thread context switch! Which thread took a page fault? –Lock/unlock must invoke kernel, but heavily used Kernel threads not portable; not offered by many OSes

17 User-Level Threads Purely inside user process; kernel oblivious Scheduler within user process for process’ own threads –In addition to kernel’s process scheduler User-level scheduler must –Know when thread makes blocking syscall –Not block process; switch to another thread –Know when I/O done, to wake up original thread

18 User-Level Thread Implementation Kernel User Space Filesystem Disk Driver Hardware App1App2 00 N M t1 stack t2 stack Thread Scheduler Process Scheduler

19 User-Level Threads: Details Apps linked against thread library Library contains “fake” read(), write(), accept(), &c. syscalls Library can start non-blocking syscall operations Library marks threads as waiting, switches to runnable thread Kernel notifies library of I/O completion and other events; library marks waiting thread runnable

20 User-Level Threads: read() Example read() { tell kernel to start read; mark thread waiting for read; sched(); } sched() { ask kernel for I/O completion events; mark corresponding threads runnable; find runnable thread; restore registers and return; }

21 User-Level Threads: Event Notification Events thread library needs from kernel: –new network connection –data arrived on socket –disk read completed –socket ready for further write()s Resembles miniature OS inside process! Problem: user-level threads demand significant kernel support: –non-blocking system calls –uniform event delivery mechanism

22 Event Notification in Typical OSes Usually, event notification only partly supported; e.g., in UNIX: –new TCP connections, arriving TCP/pipe/tty data: YES –filesystem operation completion: NO Similarly, not all syscalls can be started without waiting, e.g., in UNIX: –connect(), read() from socket, write(): YES –open(), stat(): NO –read() from disk: SOMETIMES

23 Non-blocking System Calls: Hard to Implement Typical syscall implementation, inside the kernel, e.g., for read() (sys_read.c): sys_read(fd, user_buffer, n) { // read the file’s i-node from disk struct inode *I = alloc_inode(); start_disk(…, i); wait_for_disk(i); // the i-node tells us where the data are; read it. struct buf *b = alloc_buf(i->…); start_disk(…, b); wait_for_disk(b); copy_to_user(b, user_buffer); } Why not just return to user program instead of calling wait_for_disk()? How will kernel know where to continue? In user space? In kernel? Problem: Keeping state for complex, multi- step operations

24 User-Threads: Implementation Choices Live with only partial support for user-level threads New operating system with totally different syscall interface –One syscall per non-blocking “sub-operation” –Kernel doesn’t need to keep state across multiple steps –e.g., lookup_one_path_component() Microkernel: no system calls, just messages to servers, with non-blocking communication

25 Threads: Programming Difficulty Sharing of data structures in one address space Even on single CPU, thread model necessitates CPU concurrency –Locks often needed for mutual exclusion on data structures –May only have wanted to overlap I/O wait! Events usually occur one-at-a-time –Can we do CPU sequentially, and overlap only wait for I/O? –Yes: event-driven programming

26 Event-Driven Programming Foreshadowed by user-level threads implementation –Organize software around event arrival Write software in state-machine style –“When event X occurs, execute this function.” Library support for registering interest in events (e.g., data available to read()) Desirable properties: –Serial nature of events preserved –Programmer sees only one event/function at a time