1. Reading information from files

2. Learning Objectives By the end of this lecture, you should be able to:
Taking the text that was read in from a file and use various String functions to work with that text.
Apply the basics of arrays including loops to the text from the file.
Note: Be sure to review your arrays and loops before going over this lecture. Without a review of arrays, a major portion of this topic will be difficult to understand and apply!
Study a script that contains unfamiliar code, yet be able to modify it to suit your purposes.

3. Modifying Existing Code
As you progress, you will frequently find yourself working with code that is unfamiliar to you. This is entirely normal and is, in some ways, a "next level" of developing solid coding skills.
In this lecture, I will be showing you examples that contain new and unfamiliar code. You will not be asked to work in great deal with the unfamiliar code, but you will be expected to study the code and learn enough about it to modify it to suit your needs.
You are, of course, welcome and encouraged to try and read up on some of these new concepts on your own. Again, this is how programmers have traditionally gained new knowledge, skills, and picked up new and clever ways of accomplishing coding tasks.

4. Working with Files
Most of the world’s digital information is stored inside files. Files can include anything from a spreadsheet, database file, to a plain text file. Even digital images constitute data files that can be opened using programming code and examined or modified.
We can use JavaScript to retrieve information from a digital file and use the information in that file to interact with our user. In this lecture, we will keep it fairly simple by opening a text file and outputting its contents to a web page.
In the process, we will also review some issues pertaining to JavaScript security.

5. Scenario
Imagine that you would like to take a detailed spreadsheet and output the results to your web page.
Example
Suppose you wanted to display the results of the Kentucky Derby in an HTML table. You could manually type out all the results, but this would be painstaking, error-prone, and very inefficient! Besides, imagine if instead of a horse-race with horses, you instead were trying to print out the first 500 finishers of the Chicago Marathon!
Now consider an alternative: You find a spreadsheet of the results on the web, and save that file to your server. You can then use JavaScript to retrieve the contents of that file and display the results on your web page.

6. Example: Reading in an Excel Spreadsheet
You have been given an Excel spreadsheet with a list of the top 6 finishers at the 2015 Kentucky Derby. You want to display the results from the spreadsheet into a web document.

7. Example: Reading in an Excel Spreadsheet contd.
Here are the steps we will follow for this particular scenario:
We will take the Excel spreadsheet and convert it from Excel's xls format, to a comma-separated-value format. This means that for every line (record) in the spreadsheet, each field (e.g. first name, last name, age, etc will separated from the next field by a comma).
Each line will be read in as a single string. We will then use the String ‘split()’ function to break up this string into the individual fields.
Each time we read in a line of text and convert it to a series of values, we will output them in to our HTML document.
In the example shown here, we have outputted our results to an HTML table. (For those of you who have not seen HTML tables, don’t panic. They are not very difficult, and a < 1 hour google search and review should get you up to speed).Remember that one of the objectives is to get to a point where, with a little research and practice, you can pick up new techniques on your own.

8. Security Issue: File Access
As we will discuss in a separate lecture, one of JavaScript’s biggest security limitations is that JavaScript code is not allowed to access files on the user’s computer. However, if the user chooses a file, say, by using the HTML <input type="file"> tag, then the user has consciously given the script permission to access that file.
So we will begin by having the user select a file:
Click the button and choose the file you want to work with (e.g. enrollment.csv):<br />
<input type='file' accept='text/plain'
onchange='openFile(event)'>
The ‘onchange’ event means that when the user selects a file and clicks ‘Open’, the function ‘openFile’ will be invoked.
Also note that this is one of the situations where we require the 'event' object in our function.

9. File Access: Connecting to and opening the file
Remember that the objective here is not to try and understand all of the code. At the end of the day, what I want you to be able to do is to get the ‘gist’ of what is happening, and then to be able to modify certain key components of the code as necessary to suit your needs.
var openFile = function(event) {
//openFile function --> uses an anonymous function with an event object
var input = event.target;
//The 'target' attribute for a <input type=file> form element refers to the selected file
var reader = new FileReader(); //Don't worry about this line for now
reader.readAsText(input.files[0]);
///'reader' is a handle to the file the user selected
Don't sweat trying to completely understand this code just yet. It will make a little more sense with a little more experience.

10. File Access: Retrieving the file’s contents
We will now retrieve the file’s contents:.
In order to keep things simple, we will assume two things in this particular example:
That we are dealing with a plain-text file (i.e. it isn't an image, or an audio file, etc)
The values in the plain-text file are separated by commas (i.e. it is a 'comma delimited' file)
var fileContents = reader.result;
//'result' is a JavaScript property that retrieves ALL of the file's
//contents. They are retrieved as a single very, very long String
//We have stored this long String inside the variable 'fileContents‘
Try outputting the value of ‘fileContents’ into your HTML document (e.g. using jQuery’s append function) – you will see the entire contents of the file displayed. You will further note that even though the file may have numerous lines in it (i.e. with line breaks), you will not see these line breaks in your HTML document.

11. File Access Example input_file_as_string.htm The original text file
This program will open the file called 'short_sherlock.txt' , retrieve all of the text inside (as shown below on the left) . The file will then output that text to a web page (on the right). However, since the original input file is not properly formatted for HTML, it will not display very nicely. We will discuss some techniques for processing shortly.
The original text file
The file displayed in a browser

12. File Access: Retrieving the file’s contents
Why don't we see the line breaks?
In most plain-text files, a new line is represented by a special -- and invisible -- text character: \n
Text editors use the \n as a way of referring to a new line. However, a web browser does not recognize this character. In HTML, as we all know, new lines are indicated by <br>. Therefore, all of these \n characters will do nothing when displayed in a web browser. Instead, you will see that the browser simply display a single space between one line and the next.

13. Working with the text: Breaking up the long string into smaller strings
We have learned how to retrieve the contents of a plain text file and to store them as a long string:
var fileContents = reader.result;
This line of code now has placed the entire contents of the file inside variable called 'fileContents'. However, as we have just seen, the string is not well processed. If we tried to output that string to a web browser, we would get one long, unbroken string.
We will now break up that long string (containing the entire file) into the separate lines and paragraphs as stored in the text file. We will do this using the important string function, split().
The split() function will break up the long string into many, many smaller strings. It will store all of these strings in an array.
To break up our long string into individual lines as seen in the original text file, we will use the fact that in plain-text files, the end of a line (such as a carriage return or 'enter') is encoded with the (invisible) string character: \n
Therefore, by breaking up the string on this newline character \n , we will obtain an array of smaller strings. Here is how we will generate this array:
var lines = fileContents.split('\n');

14. var lines = fileContents.split('\n');
Breaking up the string
Let’s review what happens when we invoked split() on the newline character:
var lines = fileContents.split('\n');
Recall that the split() function breaks up a String ‘fileContents’ into numerous pieces. The argument to split() will break up the string based on that character. For eample, if we invoked split using: split(' ') it would break up the string into many strings on every space character.
In this case, because we split on the newline character, \n, each smaller string is a line (i.e. each time the user pressed 'return') in the original file.
Therefore, the variable ‘lines’ above holds an array. Each element of the array represents a line of the original text file. (Note that by "line" we mean every time the author of that file pressed enter. If word-wrap is on, then a line that wraps is not a separate line).
In other words: The first item in the variable 'lines', lines[0] holds the first line of the file (as a string). The second item in the variable 'lines', lines[1] holds the second line of the file. lines[2] holds the third line, and so on.
For example, alert( lines[7] );  would output the 8th line of the file.

15. var lines = fileContents.split('\n');
After invoking split('\n') each element of the array holds a single line
var lines = fileContents.split('\n');
lines[0] holds 'THE ADVENTURES OF'
lines[1] holds 'SHERLOCK HOLMES'
lines[2] holds a blank line
lines[3] holds 'BY'
lines[4] holds a blank line .
lines[28] holds the entire first paragraph
(since it is a single line that has wrapped)
lines[29] holds a blank line
lines[30] holds the entire second paragraph
Etc

16. Working with the text: Looping through the lines
As we have just discussed, our variable ‘lines’ is an array in which each element: lines[0], lines[1], lines[2], etc represents a line from our text file.
We will then use a loop to go through each item in our array and output each line in our array as a line in our HTML document. For example, to output the 10th line of the original text file we might say: $('#results').append( lines[9] );
But how many times do we need to loop? Clearly in this case, we want to loop once for each element in our array. But how many times is that??? Recall that every array has a property called 'length' that tells us the number of items in the array. Therefore, we will use lines.length to determine the number of times to loop.
for (var i=0; i<lines.length; i++) {
//everything inside here will occur 'numberOfLines' times
//so if our file has 10 lines, this loop will execute 10 times }

17. Looping through the array
Inside our loop, lines[i], represents a single line from our text file. The first time through the loop, 'i' has a value of 0. Therefore, the first time through the loop, lines[i] is the same thing as saying lines[0]. The second time through the loop, 'i' has a value of 1, so lines[i] is now the same thing as saying lines[1], and so on.
The code below will take each line of the file (that is every line after which the 'return' key was pressed) and output it as HTML line (i.e. separated by a <p> tag).
for (var i=0; i<lines.length; i++) {
var record = lines[i]; //The variable 'record' is holding ONE line from our file
$('#results').append( "<p>" + record + "</p>");
} //end of the 'for' loop
See: input_file_as_string_with_lines.htm

18. Working inside each individual line of text
As we know, inside our loop, lines[i], represents a single line from our text file.
Now suppose that in each individual line, we want to do some further processing of the string. For example, suppose that we had the following file (already stored in a comma-separated format) showing the results of several students. The file includes their first name, last name, birth year, enrollment year, and status (full-time v.s. part-time).
Roberta,Redgrave,1988,2013,FULL
Simon,Settle,1992,2014,PART
Tommy,Tinkerbell,1972,2011,FULL
Ursula,Underhill,1980,2015,FULL
Vladimir,Vubovik,1982,2013,PART
This time, as we go through our file line by line, we will invoke split() a second time. But this time, rather than splitting on the newline character, we will split on the comma.
For example, in the first line:
we will break it up into an array in which array[0] holds 'Roberta', array[1] holds 'Redgrave', array[2] holds 1988, and so on.

19. Working with each individual line
var lines = fileContents.split('\n');
for (var i=0; i<lines.length; i++) {
var record = lines[i]; #record holds an entire line of the original text file
recordValues = record.split(","); //Discussed next
...
As you know, each time we iterate through the loop, ‘record’ holds a line from the input file.
Recall that in our original text file, each line holds a single record separated by commas as follows:
First Name, Last Name, Birth Year, Enrollment Year, Enrollment Status
For example:
Roberta,Redgrave,1988,2013,FULL
We will therefore need use split() again -- but this time we want to break the string up based on commas:
recordValues = record.split(",");
So at this point, recordValues is an array such that the first name is stored in recordValues[0], the last name is stored in recordValues[1], the birth year is in recordValues[2], the enrollment year is in recordValues[3], and the enrollment status is in recordValues[4].

20. Working with each line of text
We will now assign the pieces of information in which we are interested (i.e. the first name, last name and enrollment year) and retrieve them from our 'recordValues' array.
Also - don’t forget that we are still working with one individual line only. The next time through the loop, we will be working with the subsequent line from the file, and so on.
var lines = fileContents.split('\n');
for (var i=0; i<lines.length; i++) {
var record = lines[i];
//record is holding a String containing one line from our text file
recordValues = record.split(",");
//Take 'record' and break it into an array, splitting on every comma
var firstName = recordValues[0];
var lastName = recordValues[1];
var enrollmentYear = recordValues[3];
//We will output this information to our HTML document…
} //end of for loop

21. Outputting the results to our HTML page
Putting it all together:
for (var i=0; i<lines.length; i++) {
var record = lines[i]; //record is holding a SINGLE line from our file
recordValues = record.split(",");
var firstName = recordValues [0];
var lastName = recordValues [1];
var enrollmentYear = recordValues [3];
//Output to the HTML document
$('#results').append ("Student #" + (i+1) + ": ");
$('#results').append ( firstName + " " + lastName +
": Enrolled in " + enrollmentYear + "<br>");
} //end of for loop

22. Recap Roberta,Redgrave,1988,2013,FULL Simon,Settle,1992,2014,PART
Tommy,Tinkerbell,1972,2011,FULL
Ursula,Underhill,1980,2015,FULL
Vladimir,Vubovik,1982,2013,PART
for (var i=0; i<lines.length; i++) {
var record = lines[i];
//The variable 'record' is holding ONE line from our file
//The first time through the loop, it is holding the first line
//The second time through the loop, it is holding the second line, etc etc
recordValues = record.split(",");
//Split the line into an array of values
//Note that we split on the comma character
var firstName = recordValues [0];
var lastName = recordValues [1];
var enrollmentYear = recordValues [3];
$('#results').append ("Student #" + (i+1) + ": ");
$('#results').append (firstName + " " + lastName + ": Enrolled in "
+ enrollmentYear + "<br>");
} //end of the 'for' loop

23. HTML Files See: parsing_input_file.htm
Then look at a slightly more elaborate example in which we output results into an HTML table:
display_file_as_table.htm