COMP7500 Advanced Operating Systems Dr. Xiao Qin Auburn University http://www.eng.auburn.edu/~xqin xqin@auburn.edu Spring, 2012

Your Background Not-A-Quiz Have you taken the operating systems class? What lab assignments have you completed in the OS class? What is your on-going dissertation or thesis research project? Is your current research project related to operating systems?

See the class webpage for the most up to date version! COMP 7500: Semester Calendar See the class webpage for the most up to date version! http://www.eng.auburn.edu/~xqin/courses/comp7500

Course Syllabus Prerequisite: COMP3500 Operating Systems 1 midterm exam and 1 final exam Grading Class Participation 10% Midterm 20% Final 20% Research Projects 40% (Two projects) Presentation 10% The syllabus is available on the course web site. Read syllabus carefully after class Prerequisite: Ask students’ background

Course Syllabus (cont.) Scale Letter grades will be awarded based on the following scale. This scale may be adjusted upwards if it is necessary based on the final grades.  A  90 B  80 C  70 D  60 F < 60 The syllabus is available on the course web site. Read syllabus carefully after class Prerequisite: Ask students’ background in assembly languages Possible penalties: Zero on assignment (minimum), failure course, note on permanent record, suspension.

Office Hours and Exams Office hours: WF 3:00-4:00pm Mid-term W 2/29/2012

Am I going to read the papers to you? NO! Papers provides a framework and complete background, so lectures can be more interactive. You do the reading We’ll discuss it Projects will go “beyond”

Questions Please ask at any time!

Focus of comp7500 Load Balancing Pretching and Caching Security Issues in Operating Systems Energy Conservation Embedded Operating Systems Performance Evaluation

Perspectives of the Computer print open() start-printer cut malloc() read-disk send save fork() track-mouse Application Software Application Software Application Software System Software System Software System Software Hardware Hardware Hardware End User View (b) Application Programmer View (c) OS Programmer View

System Software Independent of individual applications, but common to all of them Examples? C library functions A window system A database management system Resource management functions The OS

Disk Abstractions Application Programmer OS Programmer int fprintf(…) { ... write(…) … } void write() { load(…); seek(…) out(…) } load(…); seek(…); out(…); (a) Direct Control (b) write() abstraction (c) fprintf() abstraction

Abstract Resources User Interface Application Abstract Resources (API) Middleware OS Resources (OS Interface) OS Hardware Resources

Resource Sharing Space- vs time-multiplexed sharing To control sharing, must be able to isolate resources OS usually provides mechanism to isolate, then selectively allows sharing How to isolate resources How to be sure that sharing is acceptable Concurrency

Auburn University http://www.eng.auburn.edu/~xqin COMP7500 Advanced Operating Systems I/O-Aware Load Balancing Techniques Dr. Xiao Qin Auburn University http://www.eng.auburn.edu/~xqin xqin@auburn.edu Spring, 2012

Technology Trend In reality: Big Fish Eat Little Fish

Massively Parallel Processors Technology Trend 1988 Computer Food Chain Mainframe Supercomputer Mini- supercomputer Mini- computer Work- station PC Mainframe is an industry term for a large computer. Historically, a mainframe is associated with centralized rather than distributed computing. Ask a question: (1) What type of computer is ranked next to Supercomputer. (2) What type of computer has performance as high as that of Supercomputer? (MPP) High performance (supercomputers) Supercomputers – Cray T-90 Massively parallel computers – Cray T3E Balanced cost/performance Workstations – SPARCstations Servers – SGI Origin, UltraSPARC High-end PCs – Pentium quads Low cost/power Low-end PCs, laptops, PDAs – mobile Pentiums Massively Parallel Processors

Technology Trend 1998 Computer Food Chain Mini- supercomputer Mini- Clusters 1998 Computer Food Chain In terms of performance, who dominates the supercomputing market? Mainframe Work- station PC Server Supercomputer Now who is eating whom?

Symmetric Multiprocessing (SMP) Massively Parallel Processor (MPP) Supercomputer Trends in Top 500 SIMD Single processor Cluster Constellations SMP MPP www.top500.org Nov. 2004 Parallel Computing Architectures in Top 500 Job market in Cluster Computing Grid and Distributed Computing SGI workstation is SMP Gray T3E is MPP CPU M MEMORY BUS/CROSSBAR CPU PC network Symmetric Multiprocessing (SMP) Massively Parallel Processor (MPP) cluster

Growth in Microprocessor Performance 1.6GHz 2002, 2.16GHz Observations?

Six Generations of DRAMs Tell my story of buy the first PC in China when I was Junior student $1000 among which $300 is for 4MB memory chips

Technology  dramatic change Processor transistor number in a chip: about 55% per year clock rate: about 20% per year Memory DRAM capacity: about 60% per year (4x every 3 years) Memory speed: about 10% per year Cost per bit: improves about 25% per year Disk capacity: about 60% per year Total use of data: 100% per 9 months! Network Bandwidth 10 years: 10Mb  100Mb 5 years: 100Mb  1 Gb

Updated Technology Trends (Summary) Capacity Speed (latency) Logic 4x in 4 years 2x in 3 years DRAM 4x in 3 years 2x in 10 years Disk 4x in 2 years 2x in 10 years Network (bandwidth) 10x in 5 years Updates during your study period?? BS (4 yrs) MS (2 yrs) PhD (5 yrs)

I/O-intensive Applications remote-sensing database systems long running simulations biological sequence analysis

Motivation Memory Disk Write: 32 MB/s Read : 26 MB/s Faster! PCI Bus W: 592 MB/s R: 464 MB/s C: 316 MB/s PCI Bus 264 MB/s W: 209 MB/s R: 236 MB/s disk I/O-intensive Applications require input and output of large amounts of data. I/O performance can be a potential bottleneck.

Current Solutions Disk I/O Systems Limitation Caching Prefetching Parallel I/O Limitation Low level Not Portable

Current Solutions (Cont.) Scheduling/Load balancing Space-sharing (PBS,Backfilling) Time-Sharing Centralized Control (PBS) Distributed Control Coordinated Scheduling (Gang) Non-I/O-aware (Condor, Mosix, DQS, LSF) Disk-I/O-aware Network-I/O-aware load balancing Support Sequential Jobs Support Parallel Jobs Disk-I/O Buffer Management Support Homogeneous Clusters Support Heterogeneous Clusters