May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 1 Still More Multiprogramming Issues File Systems ECEN5043 Software Engineering of Multiprogram Systems University of Colorado

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 2 Overview File Systems Multimedia-type Complications (multimedia in the sense of > 1 continuous media) Not your father’s text file format Process scheduler demands Extremely high data rates Smooth real-time play High storage needs -- (de)compression & amount

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 3 Files -- Why we need them Running process can store/retrieve limited info within its own (virtual) address space When the process terminates, the info is lost When a computer crash kills a process, the info goes Multiple programs/processes need to share info “simultaneously” Shared data must persist away

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 4 In other words, files: store a very large amount of information allow data to survive the program’s termination allow concurrent access by multiple programs

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 5 Analogy Client – server relationship is the concurrent programming analog of the relationship between the subroutine caller and the subroutine itself. Server is typically accessing a file Each client request is an independent unit Multiple requests may be handled concurrently

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 6 Client/server kinds of interactions occur within operating systems object-oriented systems networks databases etc. Common example: There is a file server module that provides operations on a file. When a client process wants to access the file, it calls the operation in the appropriate file server module

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 7 Shared-memory system File server typically implemented by a collection of subroutines and data structures that represent files (file descriptors) Interaction between a client process and a file would typically be implemented by subroutine calls. IF file is shared, important that it be written to by at most one client process at a time Shared file can safely be read concurrently

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 8 Same machine or not Clients are processes whether on same machine as server or not Shared-memory: server is implemented by collection of subroutines Subroutines programmed using mutual exclusion and condition synchronization to protect critical sections and ensure execution in appropriate orders On distributed-memory environment, server is implemented by one or more processes, often a multithreaded program, one thread per client.

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 9 Communication Semaphores or monitors Message passing, remote procedure calls, rendezvous

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 10 Operating system concerns How the files are: Structured Named Accessed Used Protected Implemented May memory-map file (by pages) -- system’s tables make the disk file the backing store for a designated memory region.

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 11 Additional file concerns Robustness inspection checklist Verify appropriate usage of Create / Open / Rename Seek / read Write / Append Close / Delete Get / Set attributes

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 12 What are concerns for multi-program systems? (reference previous two slides) What information would be important to have in a requirements spec or a design spec for a program that will be part of a multi-program system?

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 13 Multiprogramming directory surprises If execution environment is simple with simple directory, care must be taken to ensure unique file names, especially when user- selected If process changes its working directory to simplify file pathname references... what? Lots of variability in system call behavior for managing directories -- if called from within programs in a multi-program system... what?

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 14 (Blurred) Sepa- ration of concerns Users (programmers) How files are named What operations are allowed What the directory tree looks like Other interface issues Implementers (of o.s.) How files and directories are stored How disk space is managed Efficiency and reliability

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 15 Programmed Aspirin – reduces inflammation from irritation When writing software to be portable to different kinds of operating environments, how can you minimize the pain associated with differences in files, directories, etc.? Similarly, multiprogram access to shared files? If using language support for safety in accessing shared files, what is good practice?

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 16 Follow the yellow brick road... In a simple environment (no or little o.s.) Contiguous storage allocation is simple need address of the first block and the number of blocks in the file book presumes disk -- might be an in-memory “file”, might be executable in ROM in embedded system, etc. fragmentation issue -- compaction not realistic option on the fly great for CD-ROMs, DVDs, other write-once media Linked list allocation is ok with File Allocation Table for small files -- not as simple Remember text has details re index-nodes method

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 17 Shared (linked) files A multiprogram system design may consider “shared files” A shared file appears simultaneously in different directories belonging to different “users” Must solve If X appends to the file, new blocks must be visible to Y With links, files have >1 path name -- same file is found via various path “walks”

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 18 Fundamental principle of disk space management re block size What can you conclude about performance and space utilization?

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 19 Can you apply that to other performance vs. management issues?

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 20 Why worse for multiprogram systems? File system reliability -- storage reliability backups & restores -- will a restored file be where all programs expect to find it? techniques that minimize backup time tend to make recovery more complicated file system consistency -- what the text says is “elementary” compared to sophisticated storage management systems

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 21 Cache flow tag index byte# tags Example of 2-way associative cache main memory 16

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues Cache flow tags Example of 2-way associative cache main memory 16 6A250 1K

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 23 Impacts If two programs are accessing the same data in memory, what impact will the presence of a cache have? What if you have a late enhancement to create a report which includes the execution over a large trio of arrays: SumArray[ i ] = ArrayOne[ i ] + ArrayTwo[ i ] Would you be better off with three files?

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 24 Layers of complexity Typically in microprocessor world on-chip cache off-chip cache main memory secondary memory (disk) Both caches might work exactly the same -- check the caches sequentially. Main memory, however, may be managed with a paging algorithm...

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 25 On the one hand... If any essential byte in the cache is modified, it is written to disk immediately If any data byte in the cache is modified, it should be written almost as quickly UNIX -- soon (every 30 seconds) MS-DOS -- immediately Discuss the impact on performance. What are other issues to consider?

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 26 If caches are like little pages... What would be an equivalent to the Transition Lookaside Buffer for a cache? Block read ahead Assume file is sequential Reset the sequential bit if a “seek” is performed These are techniques that could help performance improvement if needed in a more primitive environment that did not already support this

May 13, 2002 ECEN5043 University of Colorado, Still more multiprogramming issues 27 CP/M ran well in 16 KB of RAM... Our purpose is generally not to understand how to write operating systems for large environments But rather, to understand issues addressed by operating systems as examples of multiprogram systems how solutions lead to new issues the simpler solutions in case they are needed in simpler environments without an adequate o.s. CP/M -- a rich source of simple solutions