ITEC400 Week Four Professor Robert D’Andrea

Agenda Administrative Issues This Week’s Expected Outcomes Moving around in UNIX Break-Out Problems Upcoming Deadlines Hands-on Information Lab Assistance, Questions, and Answers

Administrative Announcements The midterm and the final exam are both electronic. Please verify that your testing facility can administer this type of test. The Student Learning Center has been requested to send midterm exams to your proctors. They will email me within a couple days that the request has been completed. I will email each of you that your proctors should receive the SLC packets.

Course expected outcome Data disaster Rescuers Test command and link umask variable and commands Perl syntax

Unix Operation System Data Disaster Ontrack Data Recovery http://www.ontrackdatarecovery.org Data recovery Knoll Ontrack is one of the worlds largest data recovery services. They have performed more than 500K data recoveries since 1985.

Unix Operation System Data Disaster Rescuers CompuRecovery found on the Internet for desktop hard drive recovery. Datarecovery.com 7/24 found on the Internet. Provides RAID, hard drive, and laptop data recovery solutions. Norman Ken Ouchi at IBM was awarded the patent in 1978 for the recovery of data stored in failed memory unit. Later developed into RAID 10.

Unix Operation System Data Disaster Big versus small company operations Backup services off site location and media. Test data occasionally Trouble shooting

Unix Operation System Ironkey USB Flash Drive https://www.ironkey.com/demo-enterprise Data Disaster Helpful Hints - Use dedicated circuits for your connection - Keep your computer cool and in a dry place - Use a UPS (Uninterrupted Power Supply) - Don’t assume that your data is permanently destroyed even if the situation looks bad - Secure your work area and devices

Unix Operation System Test Condition if test ! -S "$1" then echo $1 does not exist or is empty. fi If the file specified by the first positional parameter to the shell procedure, $1, does not exist or is of size 0, the test command displays the message. If $1 exists and has a size greater than 0, the test command displays nothing.

Unix Operation System Test Condition test 100 -gt 99 && echo "Yes, that's true." || echo "No, that's false.“ This command will print the text "Yes, that's true." because 100 is greater than 99. test 100 -lt 99 && echo "Yes." || echo "No." This command will print the text "No." because 100 is not less than 99.

Unix Operation System Test Condition [ "awesome" = "awesome" ]; echo $? This command will print "0" because the expression is true; the two strings are identical. [ 5 -eq 6 ]; echo $? This command will print "1" because the expression is false; 5 does not equal 6.

UNIX Operating System Process and Multitasking The UNIX kernel can keep track of many processes at one time, and dividing it’s time to other system tasks. Each process submitted to the kernel is assigned a unique process ID (PID). In every version of UNIX, the PID range is 0 through 32,000 and is restrained to 5 digits.

UNIX Operating System Single Tasking Operating System A single-tasking operating systems only performs one task at a time. The user of a single-task system can switch from one window to another window, executing a different application, but only one task is ever active at a time. When a user switches from one window to another window, the switched task that is left, is suspended, until the user switches back to it. Suspended tasks are not allocated system resources, but remain as it was when it was suspended. When a suspended tasks resumes execution, it starts where it left off, as if nothing ever occurred.

UNIX Operating System

UNIX Operating System Multitasking Operating System In multitasking, only one CPU is involved, but the operating system switches from one program to another so quickly that it gives the appearance of executing all programs at the same time.

UNIX Operating System Multitasking Operating System

UNIX Operating System Multitasking Operating System There are two basic types of multitasking: preemptive and cooperative. In preemptive multitasking, the operating system parcels out CPU time slices to each program. The UNIX operating system is preemptive.

UNIX Operating System Multitasking Operating System In cooperative multitasking, each program can control the CPU for as long as it needs it. If a program is not using the CPU, however, it can allow another program to use it temporarily. OS/2, Windows 95, and Windows NT.

UNIX Operating System Multitasking versus Concurrent Processing A conventional processor can only execute a single task at a time - but by rapidly switching between tasks a multitasking operating system can make it appear as if each task is executing concurrently or at the same time.

UNIX Operating System Multitasking Processing Single-core systems are by nature multitasking, whereas multicore systems can provide true multithreading. So what is the difference between multitasking and multithreading? To examine multitasking, consider a single-core system that has four applications running, each of which spawns its own thread or process.

UNIX Operating System Multitasking Processing

UNIX Operating System Multitasking Processing Because the system has a single core, the operating system becomes the gatekeeper and assigns these four processes periodically to the core for execution as shown in the previous slide. If each process takes 25 units of time to execute, then the total execution time for all the three tasks is 100 units.

UNIX Operating System Multithreading Processing Multithreading is the ability of a program or an operating system process to manage its use by more than one user at a time and to even manage multiple requests by the same user without having to have multiple copies of the programming running in the computer.

UNIX Operating System Multithreading Process Multithreading extends the idea of multitasking into applications, so you can subdivide specific operations within a single application into individual threads. Each of the threads can run in parallel. The OS divides processing time not only among different applications, but also among each thread within an application.

UNIX Operating System

Unix Operating System What is a race condition? It is a condition where the data consistency has been lost because of the lack of coordination of the component task of a system. If one task has partially completed an updating a data structure and another task accesses the same data structure.

UNIX Operating System Examples of race conditions: 1.Banking where one person deposits money into an account balance and another person makes a withdrawal immediately. 2. Shopping online for an item and there is only one of them. Another buyer beats you to the product while you are in the process of ordering it.

UNIX Operating System Where are race conditions most prevalent? Race conditions are most common when associated with memory in applications. In computer memory or storage, a race condition may occur if commands to read and write a large amount of data are received at almost the same instant, and the machine attempts to overwrite some or all of the old data while that old data is still being read.

UNIX Operating System Where are race conditions most prevalent? The result may be one or “more of the following: a computer crash, an “illegal operation," notification and shutdown of the program, errors reading the old data or errors writing the new data. A race condition can also occur if instructions are processed in the incorrect order.

UNIX Operating System A race condition is an undesirable situation that occurs when a device or system attempts to perform two or more operations at the same time, but because of the nature of the device or system, the operations must be done in the proper sequence to be done correctly.

UNIX Operating System A race condition occurs when two or more threads can access shared data and they try to change it at the same time. Because the thread scheduling algorithm can swap between threads at any time, you don't know the order in which the threads will attempt to access the shared data. Therefore, the result of the change in data is dependent on the thread scheduling algorithm, i.e. both threads are "racing" to access/change the data. Problems often occur when one thread does a "check-then-act" (e.g. "check" if the value is X, then "act" to do something that depends on the value being X) and another thread does something to the value in between the "check" and the "act".

UNIX Operating System

UNIX Operating System Race conditions are hard to test for and detect. In some cases it took more that seven hundred iterations to fail and each run will fail at different iteration. While this never happened on my development environment, there can be cases where such test never fails. This shows how unpredictable such problems are. This is an example of a Race Condition as two threads read the same value at the same time, say both of them increment this value, and finally update the value, as shown next.

UNIX Operating System How to prevent race conditions? Locks provide synchronization. All access to the group of actions (or statements of code) is serialized and thus the threads need to take turns as the second one needs to wait for the first thread to finish.

UNIX Operating System Killing Processes At times a process must be terminated prior to its normal intended completion. The UNIX shell provides a utility named, kill to end the execution of a process. Prior to issuing the kill command to terminate a process, the processes PID must be determined by using the ps –ef command. The kill command is also used to send a preparatory signal(s) to an executing process prior to receiving a termination signal. A process can only be terminated by the owner of the process or by root. The syntax format for the kill command is as follows: kill [-options] process-ID

UNIX Operating System Parent and Child Processes Each UNIX process has two unique numbers associated with them. One is the Process ID (PID) and the second one is the Parent Process ID (PPID). All user processes in the system have a PPID, including the init process, which will be addressed in the course boot-up procedure.

UNIX Operating System Parent and Child Processes

UNIX Operating System Orphan and Zombie Processes Normally, when a child process is terminated, the parent process receives a SIGCHLD (code 17) signal from the kernel. After the parent receives the SIGCHLD signal, the parent can perform any last minute task or restart a new child process prior to the termination of its child. However, if the parent is terminated prior to its child process, the child process is left without a parent. This situation results in the child process becoming an orphan and the init process becomes its new parent process. The orphan process will then be assigned a PPID of 1.The term used to best describe the init processes action is re-parenting.

UNIX Operating System Zombie Process When a process is terminated or terminates on its own, but only partially exits the system, its presence can be displayed on the system as being in a Z state. This is a zombie, or defunct process on the system. A zombie is a process that completed execution, and is dead (walking dead). It does not consume system resources. It retains an entry in the process table. A good process display command is ps –aux.

UNIX Operating System Zombie Process

UNIX Operating System Daemon Process A daemon process is often a system related background process. Normally, a daemon process comes into being during boot-up and terminates when the system is rebooted or shutdown. A daemon process is not attached to a terminal like a foreground process. Verification of this attribute can be observed by performing the ps –ef command, and observing the “tty” field, all daemon processes display a “?” in this field.

UNIX Operating System Daemon Process Daemon processes execute in the background and their existence is under the radar screen. Users don’t know they exist unless they look for them on the system. Daemons execute waiting for data to be passed to them from some application, such as, a database, network, or printer daemon waiting for a print command. Daemon processes normal are known as service providers.

UNIX Operating System Runs in the background Has no parent process Daemon Process Runs in the background Has no parent process Has no terminal connection

UNIX Operating System Umask Calculation umask setting = 027 Complement of umask = 750 File default permission setting = 666 Directory default permission setting = 777 The resultant permissions are calculated via bitwise AND of the unary complement of the argument (using bitwise NOT) and the permissions specified by the program.

UNIX Operating System Umask Calculation

UNIX Operating System Regular Expressions Definition: Regular expression provides a concise and flexible means for "matching" (specifying and recognizing) strings of text, such as particular characters, words, or patterns of characters. Abbreviations for "regular expression" include "regex" and "regexp". The concept of regular expressions was first popularized by utilities provided by UNIX distributions, in particular the editor “ed” and the command/filter grep. Example: if ( /UNIX V5/ ) { print “UNIX V5 found on system. \n”; }

UNIX Operating System Regular Expressions Substitution operator: s/original value/new value/ Character class: Matches a single character that is contained within the brackets [ a-zA-Z0-9_ ] Special or meta characters are used to denote actions or delimit groups: + means “one or more of whatever was before the + symbol” . Matches any single character (i.e., x.z) ^ Matches the starting position within the string $ Matches the ending position of the string or the position just before a string-ending newline

UNIX Operating System Regular Expressions ^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$

UNIX Operating System What is ELF? Executable and Linking Format (ELF) is a common standard file format for executables, object code, shared libraries, and core dumps. ELF is flexible and extensible by design, and it is not bound to any particular processor or architecture. This has allowed it to be adopted by many different operating systems on many different platforms.

UNIX Operating System Perl Syntax ; End of statement delimiter , Comma for line continuation \n New line (non-printable character) <> The "diamond operator", <> is used when a program is expecting input, but isn't concerned how it arrives.

UNIX Operating System Perl Syntax $ Singular variables prefix (single value, number or string) @ Prefix for plural variable (array) % Prefix for plural variable (hash) $_ Default variable It Is the "it" variable. It's often the default parm that built-in functions use, or return into. $0 Contains the name of the program being run.

UNIX Operating System Perl Syntax while ( … ) { Action statements } What is a continuous loop? while ( 2 ) {

UNIX Operating System Perl Syntax The Perl language does not support case or switch statements. The closest way to achieve case evaluations is as follows: if ( $condition_one ) { action_one (); } elsif ( $condition_two ) { action_two (); } ... elsif { action_n (); }

UNIX Operating System Three Types of for Loops my @array; # Old style C for loops for (my $i = 0; $i < 10; $i++) { $array[$i] = $i; } # Iterating loops for my $i (@array) { print "$i\n"; }

UNIX Operating System Three Types of for Loops (continued) # Loops     print "$_\n" for @array;

UNIX Operating System Perl Syntax foreach $total (12,9,3,7) { # Sum each value in the list. $sum += $total; }

UNIX Operating System Perl Syntax foreach (-32,10,1,2,0,-1) { # Valid numbers are considered to be greater than zero # The default variable is utilized, if ( $_ > 0 ) { # Print each valid number on a single output line. print "$_\n"; }

UNIX Operating System Open Statement The open function can be used to create file handles for different purposes (input, output, and piping), you need to be able to specify which behavior you want. When you open a data file in Perl, all you have to do is specify (a) a file handle and (b) the name of the file you want to read from. open (CHECKBOOK, "checkbook.txt");

UNIX Operating System Open Functions open(file_handler, “file_name”) open(file_handler, “<file_name”) open (file_handler, “>file_name”) open (file_handler, “>>file_name”) See page 21 in your Perl text.

UNIX Operating System Filehandle Filehandle is utilized for both input and output files. Most file names are cryptic and are meaningless to programmers. The purpose of a filehandle is to help the programmer remember a simple file name throughout a program. A filehandle is a name given for a file, device, socket, or pipe. Filehandle command line format: open(filehandle, file name, permissions, chmod); Example: open($FH,$file_name);

UNIX Operating System If you want to read text from a file line-by-line, then use the following syntax: my @lines = <FILE>; The <FILE> operator - where FILE is a previously opened filehandle - returns all the unread lines of the text file in list context or a single line in scalar context. Hence, if you had a particularly large file and you wanted to conserve memory you could process it line by line: while (<FILE>) { print $_; }

UNIX Operating System What is List Processing? @math_array = (6 - 4, 4 * 4, 8 / 2, 9 - 8); while ( … ) { … }

UNIX Operating System What is a “for” Loop? for (counter = 0; counter < 10; counter++) { … } Perl's for loop, or for statement, is used to loop through a designated block of code until a specific condition is met. Three expressions are contained in a for loop: 1. Set initial state of the loop variable 2. Condition test the loop variable 3. Modify the state of the loop variable

UNIX Operating System foreach Statement Format foreach VAR (List) { … }

UNIX Operating System foreach Statement Format @myNames = ('Larry', 'Curly', 'Moe'); foreach (@myNames) { print $_; }

UNIX Operating System Perl Program Statement #!/usr/bin/perl #!/usr/bin/perl –w

UNIX Operating System Print Continuation Statement print "error: incorrect number of arguments", "\n", "usage: intlist a b (where a < b)", "\n";

UNIX Operating System Points of Interest Online Documentation Websites that have Perl documentation. The two biggest ones are: http://perldoc.perl.org/ http://search.cpan.org/ for modules

UNIX Operating System Demonstrate Perl script: ./array_display.pl ./array_sort.pl ./diamond_oper.pl <file name> `./perl_it.pl and perl_loop.pl ./read_list.pl ./sum_list.pl ./linenum.pl and ./intlist.pl ./week_four.pl

UNIX Operating System Moving Around in UNIX cal –y (display a calendar for the year) cal –j 2015 (display Julian dates) cal –m 2015 (display Monday first day) cal –s 2015 (display Sunday first day) cal 9 2015 (display September 2015 month

UNIX Operating System Moving Around in UNIX top command The top command is a useful tool for displaying processes sorted by various criteria. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND It is an interactive tool that updates frequently and displays data about physical and virtual devices, CPU usage, and load averages.

UNIX Operating System Moving Around in UNIX finger command The finger command displays information about users on a specific host. Local host finger Obtain specific information on a user on a local machine: finger dandrear View all the logged in users on a remote machine: finger @microsoft.com Get information about a specific user on a remote machine finger dandrear@microsoft.com

Break-out problems Define a Perl hash table Define ELF Define a Perl array What convenience does a Perl filehandle provide? What is the functionality of a regular expression? Are continuous loops useful? Define an object file Perl default variable Perl “it” variable Explain the s/pattern/new version/ command

Hands-On-Information Lab Assignment 4-1, Perl_Scripting_2, due June 11, 2017 Lab Assignment 5-1, Makefile_Scripting_1, due June 18, 2017 Lab Assignment 6-1, Perl_Scripting_3, due June 25, 2017 Read Chapters 4, 5, and 6 Perl Programming Read Module 7 and 8 listed under the course Web site.

After class assistance Questions? Comments? Concerns? After each Franklin Live session, I will remain on the session to provide assistance unless otherwise indicated.

Lab Assistance available by phone and/or email