Lectures 12 Files & Streams Dr. Eng. Ibrahim El-Nahry

Introduction Computer programs are associated to work with files as it helps in storing data & information permanently. File - itself a bunch of bytes stored on some storage devices. Variables and arrays are only temporary Lost during garbage collection or when a program terminates Files used for long term storage Called persistent data Stored on secondary storage devices Can store large amounts of data Much more than in variables or arrays in program memory Can “carry” data between program executions

Data Hierarchy COMPLEXITY Bit (binary digit) : either ‘1’ or ‘0’ All data represented as combination of bits Easy for electronic devices to understand Byte: eight bits Character: two bytes in C# (C# uses Unicode) Character set: set of all characters used to program and represent data on a particular computer Field: composition of characters that conveys a meaning Record: composition of several, related fields File: group of related records Record key: identifies record (ties it to a particular entity) Sequential file: typically, records stored in order of record keys Database: a collection of related files (managed by DBMS) COMPLEXITY

a collection of related files database Data Hierarchy a collection of related files database Randy Red Iris Orange Judy Green Tom Blue Sally Black file record field 01001010 byte (ASCII for J) 1 bit Data hierarchy.

DISK FILE PROGRAM Output Stream Input Stream write data to file read data from file Output Stream Input Stream data output data input PROGRAM

Files and Streams Files are sequential streams of bytes Ends with either an end-of-file marker or a specific byte When file opened C#: Creates an object Associates a stream with that object Three stream objects: Console.In: returns standard input stream object Console.Out: returns standard output stream object Console.Error: returns standard error stream object Namespace System.IO needed for file processing

Files and Streams BinaryFormatter: used to serialize and deserialize objects Serialize:converting an object into a format that can be written to a file without losing data Deserialize: Reading formatted text from a file and reconstructing the original object from it System.IO.Stream: allows representation of stream as bits FileStream: read to and write from sequential-access and random-access files MemoryStream:transfer of data directly to and from memory BufferedStream: uses buffer to transfer to memory

Files and Streams 1 2 3 4 5 6 7 8 9 n - 1 …… end of file marker C#’s view of an n-byte file.

Classes File and Directory Information stored in files Files organized in directories Directory class used to manipulate directories File class used to manipulate files Only has static methods, cannot instantiate File objects

Why to use Files: Convenient way to deal large quantities of data. Store data permanently (until file is deleted). Avoid typing data into program multiple times. Share data between programs. We need to know: - how to "connect" file to program - how to tell the program to read data - how to tell the program to write data - error checking and handling

Classes File and Directory

Classes File and Directory

public FileStream(string path, FileMode mode); Fundamental File Streaming File streaming consists of performing one of the routine operations on a file, such as creating it or opening it. This basic operation can be performed using a class called FileStream. You can use a FileStream object to get a stream ready for processing. As one of the most complete classes of file processing of the .NET Framework, FileStream is equipped with all necessary properties and methods. To use it, you must first declare a variable of it. The class is equipped with nine constructors. One of the constructors (the second) of the FileStream class has the following syntax: public FileStream(string path, FileMode mode);

example: using System; using System.IO; class Exercise {   class Exercise { static int Main() string NameOfFile = "Persons.spr";   FileStream fstPersons = new FileStream(NameOfFile, FileMode.Create); return 0; }

Open File Examples Open existing file for read and write. FileStream fileStream = new FileStream(@"c:\file.txt", FileMode.Open); Open existing file for reading. FileStream fileStream = new FileStream(@"c:\file.txt", FileMode.Open, FileAccess.Read); Open existing file for writing. FileStream fileStream = new FileStream(@"c:\file.txt", FileMode.Open, FileAccess.Write);

Open File Examples Open file for writing (with seek to end), if the file doesn't exist create it. FileStream fileStream = new FileStream(@"c:\file.txt", FileMode.Append); Create new file and open it for read and write, if the file exists overwrite it. FileStream fileStream = new FileStream(@"c:\file.txt", FileMode.Create); Create new file and open it for read and write, if the file exists throw exception. FileStream fileStream = new FileStream(@"c:\file.txt", FileMode.CreateNew);

Load Text from File to String   StreamReader streamReader = new StreamReader(filePath); string text = streamReader.ReadToEnd(); streamReader.Close(); Get Files from Directory Method Directory.GetFiles returns string array with files names (full paths). string[] filePaths = Directory.GetFiles(@"c:\MyDir\"); // returns: // "c:\MyDir\my-car.BMP" // "c:\MyDir\my-house.jpg"  

Stream Closing When you use a stream, it requests resources from the operating system and uses them while the stream is available. When you are not using the stream anymore, you should free the resources and make them available again to the operating system so that other services can use them. This is done by closing the stream. To close a stream, you can can call the Close() method of the classes

Get files from directory (with specified extension) string[] filePaths = Directory.GetFiles(@"c:\MyDir\", "*.bmp"); // returns: // "c:\MyDir\my-car.BMP" Get files from directory (including all subdirectories) string[] filePaths = Directory.GetFiles(@"c:\MyDir\", "*.bmp", SearchOption.AllDirectories); // returns: // "c:\MyDir\my-car.BMP" // "c:\MyDir\Friends\james.BMP"

Delete All Files (*.*) string[] filePaths = Directory.GetFiles(@"c:\MyDir\"); foreach (string filePath in filePaths) File.Delete(filePath);

File Position A FileStream object can reposition its file-position pointer to any position in the file. When a FileStream object is opened, its file-position pointer is set to byte position 0. You can use StreamReader property BaseStream to invoke the Seek method of the underlying FileStream to reset the file-position pointer back to the beginning of the file.

Serialization Often, you want to store and object on disk and retrieve it later This is a sufficiently common operation that support has been provided for it in the .NET framework Serialization can Create a self-describing data stream from an object Reconstitute this stream back into the object

Serialization You can serialize Any primitive Any object marked with the Serializable attribute that contains only serializable methods

Uses of Serialization Serialization is used for Saving objects to disk Transmitting objects across a network Transmitting objects as parameters of remote procedure calls

Serialization Formats Serialized data can be in several formats Binary Compact binary representation of the data XML XML representation of data SOAP Format suitable for use with web services

Serializable Classes A serializable class is marked so and contains only serializable members [Serializable] public class Person { string name; int age; Address address; … }

Serializable Classes The Address class must also be Serializable public class Address { string street; string city; string postCode; … }

Serializing an Object To serialize an object in binary, we use a BinaryFormatter Person p = new Person("Billy Bob", 42, "123 Main St.", "Toronto", "M2P 4D5"); FileStream fout = new FileStream("Person.bin", FileMode.Create); BinaryFormatter bf = new BinaryFormatter(); bf.Serialize(fout, p);

Deserializing an Object A BinaryFormatter is also used to deserialize an object FileStream fin = new FileStream("Person.bin", FileMode.Open); Person p1 = (Person)bf.Deserialize(fin);

Transient Data Transient data is data which should not be serialized Data which can be calculated when the object is deserialized Static data Data which is not useful to save File objects Current date or time

Transient Data Data which is transient can be marked NonSerializable I have extended the person class to store the gender and added a field for an honorific, Mr. or Ms. The honorific can be calculated from the gender so does not need to be serialized

Transient Data To deal with transient data Mark transient fields [NonSerializable] The class must implement IDeserializable The class must provide the method public virtual void OnDeserialization( Object sender) This method will recreate missing data

Random-Access Files Allows instant access to information Individual records can be accessed directly without searching through large numbers of other records Must be implemented through the application Easiest when all records are a fixed length Data can be inserted without destroying other data in file Records can be updated without rewriting the entire file Cannot serialize Won’t guarantee a fixed-length record size

Random-Access Files 100 200 300 400 500 byte offsets 100 200 300 400 500 100 bytes byte offsets Random-access file with fixed-length records.

Creating a Random-Access File Use class BinaryWriter instead of serialization By attaching a FileStream to it, bytes can be written directly to a file

public BinaryWriter(Stream output); Stream Writing A streaming operation is typically used to create a stream. Once the stream is ready, you can write data to it. The writing operation is perform through various classes. One of these classes is BinaryWriter. The BinaryWriter class can be used to write values of primitive data types (char, int, float, double, etc). To use a BinaryWriter value, you can first declare its variable. To do this, you would use one of the class' three constructors. One of its constructors (the second) has the following syntax: public BinaryWriter(Stream output); This constructor takes as argument a Stream value, which could be a FileStream variable.

Example using System; using System.IO; class Exercise { static int Main() string NameOfFile = "Persons.spr";   FileStream fstPersons = new FileStream(NameOfFile, FileMode.Create); BinaryWriter wrtPersons = new BinaryWriter(fstPersons);   wrtPersons.Write("James Bloch"); wrtPersons.Write("Catherina Wallace"); wrtPersons.Write("Bruce Lamont"); wrtPersons.Write("Douglas Truth"); wrtPersons.Close(); fstPersons.Close(); return 0; }

Reading Data Sequentially from a Random-Access File Use class BinaryReader instead of deserialization By attaching a FileStream to it, bytes can be read directly from a file

public BinaryReader(Stream input); Stream Reading Once the stream is ready, you can get prepared to read data from it. To support this, you can use the BinaryReader class. This class provides two constructors. One of the constructors (the first) has the following syntax: public BinaryReader(Stream input); This constructor takes as argument a Stream value, which could be a FileStream object. After declaring a FileStream variable using this constructor, you can read data from it. To do this, you can call an appropriate method. This class provides an appropriate method for each primitive data type. After using the stream, you should close it to reclaim the resources it was using. This is done by calling the Close() method.

 using System; using System.IO;  class Exercise { static int Main() string NameOfFile = "Persons.spr"; FileStream fstPersons = new FileStream(NameOfFile, FileMode.Create); BinaryWriter wrtPersons = new BinaryWriter(fstPersons);   wrtPersons.Write("James Bloch"); wrtPersons.Write("Catherina Wallace"); wrtPersons.Write("Bruce Lamont"); wrtPersons.Write("Douglas Truth"); wrtPersons.Close(); fstPersons.Close(); FileStream fstPersons = new FileStream(NameOfFile, FileMode.Open); BinaryReader rdrPersons = new BinaryReader(fstPersons);

string strLine = null; strLine = rdrPersons.ReadString(); Console.WriteLine(strLine); rdrPersons.Close(); fstPersons.Close(); return 0; }

Example using System; using System Example using System; using System.IO; public class FileCopy {     public static void Main(string[] args)     {         if (args.Length < 2)         {             //Show usage information if incorrect number             //of parameters has been entered             Console.WriteLine("Usage: FileCopy <SourceFile><DestinationFile>");             return;         }

        try         {             //Open a FileStream to the source file             FileStream fin = new FileStream(args[0], FileMode.Open,             FileAccess.Read, FileShare.Read);             //Open a FileStream to the destination file             FileStream fout = new FileStream(args[1], FileMode.OpenOrCreate,             FileAccess.Write, FileShare.None);             //Create a byte array to act as a buffer             Byte[] buffer = new Byte[32];             Console.WriteLine("File Copy Started");             //Loop until end of file is not reached             while (fin.Position != fin.Length)             {                 //Read from the source file                 //The Read method returns the number of bytes read                 int n = fin.Read(buffer, 0, buffer.Length);

//Write the contents of the buffer to the destination file fout                 //Write the contents of the buffer to the destination file                 fout.Write(buffer, 0, n);             }             //Flush the contents of the buffer to the file             fout.Flush();             //Close the streams and free the resources             fin.Close();             fout.Close();             Console.WriteLine("File Copy Ended");          }

catch (IOException e) { //Catch a IOException Console         catch (IOException e)         {             //Catch a IOException             Console.WriteLine("An IOException Occurred :" + e);         }         catch (Exception e)         {             //Catch any other exception that occurs             Console.WriteLine("An Exception Occurred :" + oe);         }         Console.ReadLine();     } }