1. Programming for Geographical Information Analysis: Core Skills
Lecture 7:Core Packages:
File Input/Output
Welcome to ‘Programming for Geographical Information Analysis: Core Skills’.

2. This lecture
Files Text files Binary files

3. Files
File types
Dealing with files starts with encapsulating the idea of a file in an object

4. File locations
Captured in two classes: java.io.File Encapsulates a file on a drive. java.net.URL Encapsulates a Uniform Resource Locator (URL), which could include internet addresses.
Today we will mainly be dealing with Files, rather than URLs.

5. java.io.File
Before we can read or write files we need to capture them. The File class represents an external file. File(String pathname); File f = new File("e:/myFile.txt"); However, we must remember that different OSs have different file systems. Note the use of a forward slash. Java copes with most of this, but “e:” wouldn’t work in *NIX / Mac / mobiles etc.
It’s easier to use a forward slash than a backslash – backslashes are an escape character – i.e. they mean than the next character isn’t to be interpreted literally, e.g. \n is a newline. The backslash therefore needs to be written \\ if you’re going to use it.
What we need is either a user defined or a relative file path. We dealt with this in practical one.

6. Getting file locations
java.awt.FileDialog
Opens a “Open file” box with a directory tree in it. This stays open until the user chooses a file or cancels.
Once chosen use FileDialog’s getDirectory() and getFile() methods to get the directory and filename.
Remember that the user could have cancelled, so check that the directory and file aren’t “null”.

7. Getting file locations
import java.awt.*; import java.io.*; FileDialog fd = new FileDialog(new Frame()); fd.setVisible(true); File f = null; if((fd.getDirectory() != null)||( fd.getFile() != null)) { f = new File(fd.getDirectory() + fd.getFile()); }
We check if the directory and file are “null”, because this is what they’d be if the user pushed “Cancel”.
Note the split in label creation and object attachment. What is the value of ‘f’ if the if statement doesn’t run? Null.
Note the “null” passed in to the constructor – this is to replace an awt class we haven’t yet looked at.

8. The application directory
Each object has a java.lang.Class object associated with it. This represents the class loaded into the JVM. One use is to get resources local to the class, i.e. in the same directory as the .class file. We use a java.net.URL object to do this. Class thisClass = getClass(); URL url = thisClass.getResource("myFile.txt"); We can then use URL’s getPath() to return the file path as a String for the File constructor.
You can try just using the filename if the file is in the directory you’re in, however, it’s a bit iffy, and the method above allows you to build relative filenames.
We’ll see an easier method using Class in a bit.
More on URL objects later in the course.

9. Useful File methods createNewFile() and createTempFile()
exists(), canRead() and canWrite()
Test whether the file exists and can be read or written to.
createNewFile() and createTempFile()
Create a new file, and create a new file in “temp” or “tmp”.
delete() and deleteOnExit()
Delete the file (if permissions are correct). Delete when JVM shutsdown.
isDirectory() and listFiles()
Checks whether the File is a directory, and returns an array of Files representing the files in the directory. Can use a FilenameFilter object to limit the returned Files.
File can have read, write and execute permissions. You usually only see the read/write permissions in Windows.
Also, there’s a setReadOnly(): This Method makes the file or directory read-only.

10. Files
File types
As we’ll see, the type of the file has a big effect on how we handle it.

11. Binary vs. Text files
All files are really just binary 0 and 1 bits. In ‘binary’ files, data is stored in binary representations of the primitive types: = int = int = int = int = int = int = int 255
8 bits = 1 byte
For reference, Java is Bigendian and two’s complement signed, which means that ints are arranged in memory thus:
= 1
Rather than
= 1
And that they use the ‘two’s complement’ method for making negatives:

12. Binary vs. Text files
In text files, which can be read in notepad++ etc. characters are stored in smaller 2-byte areas by code number: = code 65 = char “A” = code 97 = char “a”

13. Characters
All chars are part of a set of 16 bit international characters called Unicode.
These extend the American Standard Code for Information Interchange (ASCII) , which are represented by the ints 0 to 127, and its superset, the 8 bit ISO-Latin 1 character set (0 to 255).
There are some invisible characters used for things like the end of lines.
char back = 8; // Try 7, as well!
System.out.println("hello" + back + "world");
The easiest way to use stuff like newline characters is to use escape characters.
System.out.println("hello\nworld");
The ASCII codes are on the website.

14. Binary vs. Text files Note that :
= code 49 = char “1”
Seems much smaller – it only uses 2 bytes to store the character “1”, whereas storing the int 1 takes 4 bytes.
However each character takes this, so:
= code 49 = char “1”
= code 49, 50 = char “1” “2”
= code 49, 50, 55 = char “1” “2” “7”
Whereas :
= int 127
ints take four bytes max to store, whereas, if we stored each digit as a character it would take two bytes per character, so if the number is over 2 digits it’s better to use binary storage.
A complete list of codes can be found at:

15. Binary vs. Text files
In short, it is much more efficient to store anything with a lot of numbers as binary (not text). However, as disk space is cheap, networks fast, and it is useful to be able to read data in notepad etc. increasingly people are using text formats like XML. As we’ll see, the filetype determines how we deal with files.

16. Review
File f = new File("e:/myFile.txt"); Three methods of getting file locations: Hardwiring FileDialog Class getResource() Need to decide the kind of file we want to deal with.

17. This lecture
Files
Text files
Binary files

18. Input and Output (I/O)
So, how do we deal with files (and other types of I/O)? In Java we use address encapsulating objects, and input and output “Streams”. Streams are objects which represent the external resources which we can read or write to or from. We don’t need to worry about “how”. Input Streams are used to get stuff into the program. Output streams are used to output from the program.
We won’t deal with Java’s communication with hardware as this is a subject beyond the level of this course.
Note that we are looking at input and output imagining we are in the program. The output from another program is pulled into our program using an input stream, not an output stream.

19. Streams
Streams based on four abstract classes… java.io.Reader and Writer Work on character streams – that is, treat everything like it’s going to be a character. java.io.InputStream and OutputStream Work on byte streams – that is, treat everything like it’s binary data.

20. Character based streams
Two abstract superclasses – Reader and Writer.
These are used for a variety of character streams.
Most important are:
FileReader : for reading files.
FileWriter : for writing files.

21. Example
File f = new File(“myFile.txt"); FileReader fr = null; try { fr = new FileReader (f); } catch (FileNotFoundException fnfe) { fnfe.printStackTrace(); } char char1 = fr.read(); fr.close(); } catch (IOException ioe) { ioe.printStackTrace();
Read one character out
of the file.
Note the different exceptions that have to be caught.
Closing the file is important and polite. You may have found yourself that you’ve used a file in one program, and then not been able to use it in another until you’ve rebooted your machine – this is because some programmer hasn’t politely release the file back to the operating system.
Close the connection to
the file so others can use it.

22. Example File f = new File("myFile.txt"); FileWriter fw = null; try {
fw = new FileWriter (f, true);
} catch (IOException ioe) {
ioe.printStackTrace(); }
fw.write("A");
fw.flush();
fw.close();
Note this boolean is optional and sets whether to append to the file (true) or overwrite it (false). Default is overwrite.
Here we see the constructor for FileWriter is polymorphic – we can either send in a boolean or leave it out.
Make sure everything in the stream is written out.

23. Buffers Plainly it is a pain to read a character at a time.
It is also possible that the filesystem may be slow or intermittent, which causes issues.
It is common to wrap streams in buffer streams to cope with these two issues.
BufferedReader br = new BufferedReader(fr);
BufferedWriter bw = new BufferedWriter(fw);

24. Example
BufferedReader br = new BufferedReader(fr); // Remember fr is a FileReader not a File. int lines = -1; String textIn = " "; String[] file = null; try { while (textIn != null) { textIn = br.readLine(); lines++; } file = new String[lines]; // close the buffer here and remake both FileReader and // buffer to set it back to the file start. for (int i = 0; i < lines; i++) { file[i] = br.readLine(); br.close(); } catch (IOException ioe) {}
Run through the file once to count the lines and make a String array the right size.
Here’s the (nearly) full code for reading a text file into an array of Strings, one line per String (‘nearly’ because we’ve left out the exception catching).
Note that we start lines as -1 because the last line is a null but still increases lines by one.
Note that all the variables are created outside the try block to avoid scoping issues.
Go back to the start of the file and read it into the array.

25. Example
String[][] strData = getStringArray(); BufferedWriter bw = new BufferedWriter (fw); // Remember fw is a FileWriter not a File. try{ for (int i = 0; i < strData.length; i++) { for (int j = 0; j < strData[i].length; j++) { bw.write(strData[i][j] + ", "); } bw.newLine(); bw.close(); } catch (IOException ioe) {}
Here’s the (nearly) full code for reading a text file into an array of Strings, one line per String (‘nearly’ because we’ve left out the exception catching).

26. Processing data
This is fine for text, but what if we want values and we have text representations of the values? There is a difference between 0.5 and “0.5”. The computer understands the first as a number, but not the second First, parse (split and process) the file to get each individual String representing the numbers. Second, turn the text in the file into real numbers.

27. java.util.StringTokenizer
String line = “Call me Dave”;
StringTokenizer st = new StringTokenizer(line);
while (st.hasMoreTokens()) { System.out.println(st.nextToken()); }
prints the following output:
Call me
Dave
Default separators: space, tab, newline, carriage-return character, and form-feed.
There is actually also a kind of buffered stream that will do the same job, called a StreamTokeniser.
Used to break character streams up into “tokens”, which may, for example, be words.
Reader r = new BufferedReader (new InputStreamReader(is));
StreamTokenizer st = new StreamTokenizer(r);
st.resetSyntax();
st.whitespaceChars(0, “ “);
st.nextToken();
String firstWord = st.toString();

28. Processing data
There are wrapper classes for each primitive that will do the cast: double d = Double.parseDouble("0.5"); int i = Integer.parseInt("1"); boolean b = Boolean.parseBoolean("true"); On the other hand, for writing, String can convert most things to itself: String str = String.valueOf(0.5); String str = String.valueOf(data[i][j]);

29. Example
for (int i = 0; i <= lines; i++) { file[i] = br.readln(); } br.close(); double[][] data = new double [lines][]; for (int i = 0; i < lines; i++) { StringTokenizer st = new StringTokenizer(file[i],", "); data[i] = new double[st.countTokens()]; int j = 0; while (st.hasMoreTokens()) { data[i][j] = Double.parseDouble(st.nextToken()); j++;
Comma and space separated
data
Here’s the (nearly) full code for reading a text file into an array of Strings, one line per String (‘nearly’ because we’ve left out the exception catching).
This code starts as the code before for reading strings.

30. Example
double[][] dataIn = getdata(); BufferedWriter bw = new BufferedWriter (fw); String tempStr = ""; try { for (int i = 0; i < dataIn.length; i++) { for (int j = 0; j < dataIn[i].length; j++) { tempStr = String.valueOf(dataIn[i][j]); bw.write(tempStr + ", "); } bw.newLine(); bw.close(); } catch (IOException ioe) {}
Here’s the (nearly) full code for reading a text file into an array of Strings, one line per String (‘nearly’ because we’ve left out the exception catching).
Note that we make tempStr outside of the loops so the label is reused, rather than inefficiently being recreated each time.
Converts the double to a String.

31. java.util.Scanner Wraps around all this to make reading easy:
Scanner s = null;
try {
s = new Scanner(
new BufferedReader(
new FileReader("myText.txt")));
while (s.hasNext()) {
System.out.println(s.next()); }
if (s != null) {
s.close();
} catch (Exception e) {}
However, no token counter, so not great for reading into arrays.
Code from:

32. Scanners By default looks for spaces to tokenise on.
Can set up a regular expression to look for.
Comma followed by optional space:
s.useDelimiter(",\\s*");

33. Data conversion
s.next() / s.hasNext() String nextBoolean() / hasNextBoolean() boolean nextDouble() / hasNextDouble() double nextInt() / hasNextInt() int nextLine() / hasNextLine() String If the type doesn’t match, throws InputMismatchException.

34. Reading from keyboard
Scanner s = new Scanner(System.in); int i = s.nextInt(); String str = s.nextLine();

35. Parsing Strings
Usually with text we want to extract useful information. Search and replace.

36. String searches Returns a boolean.
startsWith(String prefix), endsWith(String suffix)
Returns a boolean.
indexOf(int ch), indexOf(int ch, int fromIndex)
Returns an int representing the first position of the first instance of a given Unicode character integer to find.
indexOf(String str),
indexOf(String str, int fromIndex)
Returns an int representing the position of the first instance of a given String to find.
lastIndexOf
Same as indexOf, but last rather than first.

37. String manipulation Replaces one character with another.
replace(char oldChar, char newChar)
Replaces one character with another.
substring(int beginIndex, int endIndex)
substring(int beginIndex)
Pulls out part of the String and returns it.
toLowerCase(), toUpperCase()
Changes the case of the String.
trim()
Cuts white space off the front and back of a String.

38. Example str now “old pond; frog jumping; splash”
String str = "old pond; frog leaping; splash";
int start = str.indexOf("leaping");
int end = str.indexOf(";", start);
String startStr = str.substring(0, start);
String endStr = str.substring(end);
str = startStr + "jumping" + endStr;
str now “old pond; frog jumping; splash”
The line is Matsuo Bashô’s Frog Haiku. You can find the translation here:

39. Review
Use a java.util.Scanner where possible. Otherwise use a FileWriter/Reader. But remember to buffer both.

40. This lecture
Files
Text files
Binary files

41. Byte streams
InputStream Read methods return -1 at the end of the resource. FileInputStream(File fileObject) Allows us to read bytes from a file. OutputStream Used to write to resources. FileOutputStream(File fileObject) Used to write to a file if the user has permission. Overwrites old material in file. FileOutputStream(File fileObject, boolean append) Only overwrites if append is false.
Note that streams return –1 at the end of reading, they don’t actually read –1 into themselves.

42. Example
FileInputStream ourStream = null; File f = new File(“e:/myFile.bin”); try { ourStream = new FileInputStream(f); } catch (FileNotFoundException fnfe) { // Do something. } The Stream is then usually used in the following fashion: int c = 0; while( (c = ourStream.read()) >= 0 ) { // Add c to a byte array (more on this shortly). ourStream.close();
Note that the single byte of binary data is stored temporarily in the int variable, which has space for four bytes. All four are only filled when the -1 is returned at the end of the file. The system can then recognise this as the number “-1”. Up until that point, even if it reads the number “-1” from the file, it will get pulled into four pieces before hitting the int variable, so it will never trigger the stop condition.
This isn’t the place to go into the “Two's Complement” mechanism of storing negative numbers, but if you are interested you can find details here:

43. Byte streams II
There are cases where we want to write to and from arrays using streams. These are usually used as a convenient way of reading and writing a byte array from other streams and over the network. ByteArrayInputStream ByteArrayOutputStream
You may think these are a bit obscure, but they have one very good use...

44. Example
FileInputStream fin = new FileInputStream(file);
ByteArrayOutputStream baos = new
ByteArrayOutputStream();
int c;
while((c = fin.read()) >= 0) {
baos.write(c); }
byte[] b = baos.toByteArray();
Saves us having to find out size of byte array as ByteArrayOutputStream has a toByteArray() method.

45. Buffering streams
As with the FileReader/FileWriter: BufferedInputStream BufferedOutputStream You wrap the classes using the buffer’s constructors.

46. Other byte streams
RandomAccessFile Used for reading and writing to files when you need to write into the middle of files as opposed to the end. PrintStream Was used in Java 1.0 to write characters, but didn’t do a very good job of it. Now deprecated as an object, with the exception of System.out, which is a static final object of this type. Object Streams

47. Serialization
Given that we can read and write bytes to streams, there’s nothing to stop us writing objects themselves from the memory to a stream. This lets us transmit objects across the network and save the state of objects in files. This is known as object serialization. More details at:

48. Summary
We can represent and explore the files on a machine with the File class. To save us having to understand how external info is produced, java uses streams. We can read and write bytes to files or arrays. We can store or send objects using streams. We can read and write characters to files or arrays. We should always try and use buffers around our streams to ensure access.