1. CSE 501N Fall ‘09 18: Files and Streams 06 November 2009 Nick Leidenfrost

2. 2 Lecture Outline Storing data to the hard disk  Files  Streams  Storage Decisions  Serialization

3. 3 File System File Hierarchy In general, files in computers are organized in a directory tree  A directory is a virtual container that holds files and other directories  A.k.a folder

4. 4 Files Naming Conventions Filenames are unique and case-sensitive Extensions serve as a “hint” or “shortcut” for the type of data contained in the file  For Us  For the Operating System (OS) H:/workspace/Lab6/Ship.java /home/username/workspace/Lab6/Ship.java Windows Mac / Linux / Unix drive path filename extension “full path”

5. 5 Referring to Files in Programs Absolute vs. Relative Paths Absolute Path  The “full path” to the file Relative Path  A path that specifies a file’s location relative to another location “another location” = the location of our program H:/workspace/Lab6/images/mothership.gif images/Ship.java

6. 6 Relative Paths./ And../ Some special notation specific to relative paths lets us refer to our own directory, as well as our parent directory./ and../ ./ = The current directory ../ = The parent directory of the current directory (or the directory above the current directory) This notation is fairly standard in computing [ cmd example ]

7. 7 Relative Paths Examples Relative Paths  Inside our directory  Inside a subdirectory  In our “parent” directory  In a “sibling directory”  In directory above our parent directory images/mothership.gif images/gif/small/mothership.gif../motherhsip.gif../../../motherhsip.gif mothership.gif./mothership.gif../images/motherhsip.gif

8. 8 Absolute vs. Relative Absolute  Path will rely on exactly the same directory structure being in place on every computer  Application can be moved independently of resources, and resources will still be found  Generally not portable (System-specific)  Usually set on installation Relative  Will be correct as long as resources stay in the same place relative to the application  Application and resources can be moved and still function From computer to computer From directory to directory on the same computer

9. 9 File Formats Text and Binary Two ways to store data:  Text format (a.k.a. plain-text) Data stored as characters Human readable Less efficient with respect to storage A.k.a. ASCII (ask · ee)  (American Standard Code For Information Interchange)  Binary format Data stored as bytes Looks like gibberish to Humans Relies on a defined structure More compact / efficient than plain-text // Let’s look at some examples!

10. 10 ASCII Format Let’s look at storing an integer in a plain- text file  An int in Java is 4 bytes  We want to store the number 12,345  Our file actually holds the character ‘1’, followed by ‘2’, then ‘3’, ‘4’ and finally, ‘5’ This takes at least 5 bytes

11. 11 Binary Format Data items are represented in bytes Integer 12,345 stored as a sequence of four bytes:  0 0 48 57  48*256¹ + 57*256º  Why the zeros? Why not just use 2 bytes to store it? More compact and more efficient

12. 12 Files in Java Library Support Support for file interaction (and more) can be found in Java’s java.io library  “io”: Input / Output A file is represented in the library by the File class in java.io.File We can create file objects with either absolute or relative paths // Let’s have a look at java.io.File

13. 13 Reading Text Files Simplest way to read text: use Scanner class To read from a file on disk, construct a FileReader Then, use the FileReader to construct a Scanner object Use the Scanner methods to read data from file  next, nextLine, nextInt, and nextDouble  // Let’s look at the Scanner API FileReader reader = new FileReader("input.txt“); Scanner in = new Scanner(reader);

14. 14 Writing Text Files To write to a file, construct a PrintWriter object  // Let’s look at the PrintWriter API If file already exists, it is emptied before the new data are written into it If file doesn't exist, an empty file is created PrintWriter out = new PrintWriter("output.txt");

15. 15 Writing Text Files Use print and println to write into a PrintWriter : You must close a file when you are done processing it: Otherwise, not all of the output may be written to the disk file out.println(29.95); out.println(new Rectangle(5, 10, 15, 25)); out.println("Hello, World!"); out.close();

16. 16 A Sample Program Reads all lines of a file and sends them to the output file, preceded by line numbers Sample input file: Mary had a little lamb Whose fleece was white as snow. And everywhere that Mary went, The lamb was sure to go!

17. 17 A Sample Program Program produces the output file: (Program could be used for numbering Java source files, etc.) /* 1 */ Mary had a little lamb /* 2 */ Whose fleece was white as snow. /* 3 */ And everywhere that Mary went, /* 4 */ The lamb was sure to go!

18. 18 Write the code for this program // Code Example: FileNumberer.java

19. 19 File LineNumberer.java import java.io.FileReader; import java.io.IOException; import java.io.PrintWriter; import java.util.Scanner; public class FileNumberer { public static void main(String[] args) { Scanner console = new Scanner(System.in); System.out.print("Input file: "); String inputFileName = console.next(); System.out.print("Output file: "); String outputFileName = console.next(); try {

20. 20 File LineNumberer.java FileReader reader = new FileReader(inputFileName); Scanner in = new Scanner(reader); PrintWriter out = new PrintWriter(outputFileName); int lineNumber = 1; while (in.hasNextLine()) { String line = in.nextLine(); out.println("/* " + lineNumber + " */ " + line); lineNumber++; } out.close(); } catch (IOException exception) {

21. 21 File LineNumberer.java System.out.println("Error processing file:" + exception); }

22. 22 File Dialog Boxes More user friendly way of selecting files // Let’s integrate this into our FileNumberer

23. 23 File Dialog Boxes JFileChooser chooser = new JFileChooser(); FileReader in = null; int choiceResult = chooser.showOpenDialog(null); if (choiceResult == JFileChooser.APPROVE_OPTION) { File selectedFile = chooser.getSelectedFile(); reader = new FileReader(selectedFile);... }

24. 24 Text Format Human-readable form Data stored as sequence of characters  Integer 12345 stored as characters ' 1 ' ' 2 ' ' 3 ' ' 4 ' ' 5 ' Use Reader and Writer and their subclasses to process input and output To read: To write FileReader reader = new FileReader("input.txt"); FileWriter writer = new FileWriter("output.txt");

25. 25 Binary Format Reading and writing binary files  Use subclasses of InputStream and OutputStream To read: To write FileInputStream inputStream = new FileInputStream("input.bin"); FileOutputStream outputStream = new FileOutputStream("output.bin");

26. 26 Streams: Input and Output We read from an InputStream We write to an OutputStream  As with System.out  Imagine the “Stream” as a hose connecting the data source and the data destination Output (writing) Input (reading) The stream

27. 27 Reading a Single Character from a File in Text Format Use various read methods in InputStream class to read a single byte / array of bytes  returns the next byte as an int Returned value 0 <= x <= 255  or the integer -1 at end of file InputStream in =...; int next = in.read(); byte b; if (next != -1) b = (byte) next;

28. 28 Text and Binary Format Use variations of the write method to write a single byte / array of bytes read and write are the only input and output methods provided by the file input and output classes Java stream package principle: each class should have a very focused responsibility  Use Library of subclasses for more high-level behavior

29. 29 Text and Binary Format Job of InputStream / OutputStream : interact with data sources and get bytes To read numbers, strings, or other objects, combine a Stream with other classes  E.g. java.util.Scanner

30. 30 File Example A Simple Encryption Program File encryption  To scramble it so that it is readable only to those who know the encryption method and secret keyword To use Caesar cipher  Choose an encryption key–a number between 1 and 25  Example: If the key is 3, replace A with D, B with E,...

31. 31 An Encryption Program Example text: To decrypt, use the negative of the encryption key

32. 32 Code for this? // Code Example: FileEncryptor.java

33. 33 File Encryptor.java import java.io.File; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.InputStream; import java.io.OutputStream; import java.io.IOException; /** An encryptor encrypts files using the Caesar cipher. For decryption, use an encryptor whose key is the negative of the encryption key. */ public class Encryptor { /** Constructs an encryptor. @param aKey the encryption key */

34. 34 File Encryptor.java protected int key; public Encryptor(int aKey) { this.key = aKey; } /** Encrypts the contents of a file. @param inFile the input file @param outFile the output file */ public void encryptFile(String inFile, String outFile) throws IOException { InputStream in = null; OutputStream out = null; try {

35. 35 File Encryptor.java in = new FileInputStream(inFile); out = new FileOutputStream(outFile); encryptStream(in, out); } finally { if (in != null) in.close(); if (out != null) out.close(); } /** Encrypts the contents of a stream. @param in the input stream @param out the output stream */

36. 36 File Encryptor.java public void encryptStream(InputStream in, OutputStream out) throws IOException { boolean done = false; while (!done) { int next = in.read(); if (next == -1) done = true; else { byte b = (byte) next; byte c = encrypt(b); out.write(c); }

37. 37 File Encryptor.java /** Encrypts a byte. @param b the byte to encrypt @return the encrypted byte */ public byte encrypt(byte b) { return (byte) (b + key); } private int key; }

38. 38 Storage Options Random Access vs. Sequential Access Sequential access  A file is processed a byte at a time  It can be inefficient Random access  Allows access at arbitrary locations in the file  Only files on disk support random access System.in and System.out (normal input and output streams) do not  Each disk file has a special file pointer position You can read or write at the position where the file pointer is

39. 39 Storage Options Random Access vs. Sequential Access Each disk file has a special file pointer position  You can read or write at the position where the pointer is

40. 40 RandomAccessFile You can open a file either for  Reading only (" r ")  Reading and writing (" rw ") To move the file pointer to a specific byte (moves file pointer to nth byte) RandomAccessFile f = new RandomAcessFile("bank.dat","rw"); f.seek(n);

41. 41 RandomAccessFile To get the current position of the file pointer: To find the number of bytes in a file: long n = f.getFilePointer(); // of type "long" because files can be very large long fileLength = f.length();

42. 42 Random Access A Sample Program Use a random access file to store a set of bank accounts Program lets you pick an account and deposit money into it To manipulate a data set in a file, pay special attention to data formatting  Suppose we store the data as text Say account 1001 has a balance of $900, and account 1015 has a balance of 0

43. 43 Random Access A Sample Program What if we want to deposit $100 into account 1001? If we now simply write out the new value, the result is

44. 44 Random Access A Sample Program What if money becomes too big?  This is caused by one of the downsides of human- readable file formats Better way to manipulate a data set in a file:  Give each value a fixed size that is sufficiently large  Every record has the same size  Easy to skip quickly to a given record  To store numbers, use binary format for scalability

45. 45 Random Access A Sample Program RandomAccessFile class stores binary data readInt and writeInt read/write integers as four-byte quantities readDouble and writeDouble use 8 bytes double x = f.readDouble(); f.writeDouble(x);

46. 46 Random Access A Sample Program To find out how many bank accounts are in the file public int numAccounts () throws IOException { return (int) (file.length() / RECORD_SIZE); // RECORD_SIZE is 12 bytes: // 4 bytes for the account number and // 8 bytes for the balance }

47. 47 Random Access A Sample Program To read the nth account in the file public BankAccount read (int n) throws IOException { file.seek(n * RECORD_SIZE); int accountNumber = file.readInt(); double balance = file.readDouble(); return new BankAccount(accountNumber, balance); }

48. 48 Random Access A Sample Program To write the nth account in the file public void writeNth (int n, BankAccount account) throws IOException { file.seek(n * RECORD_SIZE); file.writeInt(account.getAccountNumber()); file.writeDouble(account.getBalance()); }

49. 49 Object Streams Reading and Writing Objects? WTF? Writing Objects directly to streams ObjectOutputStream class can write a entire objects to disk ObjectInputStream class can read objects back in from disk Objects are saved in binary format; hence, you use streams // Let’s look at the APIs

50. 50 Writing a BankAccount Object to a File The object output stream saves all instance variables BankAccount b =...; OutputStream os = new FileOutputStream("bank.dat"); ObjectOutputStream out = new ObjectOutputStream(fos); out.writeObject(b);

51. 51 Reading a BankAccount Object From a File readObject returns an Object reference Hence, we must remember the types of the objects that you saved and use a cast InputStream is = new FileInputStream("bank.dat"); ObjectInputStream in = new ObjectInputStream(is); BankAccount b = (BankAccount) in.readObject();

52. 52 Reading a BankAccount Object From a File readObject method can throw a ClassNotFoundException  Why is this? It is a checked exception You must catch or declare it

53. 53 Writing Complex Objects Write and Read an ArrayList to a File Write Read ArrayList bl = new ArrayList (); // Now add many BankAccount objects into bl out.writeObject(bl); ArrayList bl = (ArrayList ) in.readObject();

54. 54 Serializable Objects that are written to an object stream must belong to a class that implements the Serializable interface. Serializable interface has no methods.  What is it good for then!? class BankAccount implements Serializable {... }

55. 55 Serializable Implementing Serializable tells Java that a class can be serialized  Most issues of serialization cannot be: Entirely identified with an interface Detected by the compiler  Therefore Java is forced to trust us The interface has only Semantic meaning Exceptions will be thrown if problems arise If you want more control over serialization, implement java.io.Externizable

56. 56 Serializable Serialization: process of saving objects to a stream  Each object is assigned a serial number on the stream  If the same object is saved twice, only serial number is written out the second time  When reading, duplicate serial numbers are restored as references to the same object

57. 57 Conclusion Questions? I will be in Lab now until 6:30