These notes are intended for use by students in CS0445 at the University of Pittsburgh and no one else These notes are provided free of charge and may.

These notes are intended for use by students in CS0445 at the University of Pittsburgh and no one else These notes are provided free of charge and may not be sold in any shape or form Material from these notes is obtained from various sources, including, but not limited to, the following: Data Structures and Abstractions with Java, 2nd, 3rd and 4th Editions by Frank Carrano (and Timothy Henry) Data Structures and the Java Collections Framework by William Collins Classic Data Structures in Java by Timothy Budd Java By Dissection by Pohl and McDowell Java Software Solutions (various editions) by John Lewis and William Loftus java.sun.com and its many sub-links

Goals of Course To learn, understand and be able to utilize many of the data structures that are fundamental to computer science Data structures such as vectors, stacks, queues, linked-lists and trees are used throughout computer science We should understand these from a user's point of view: What are these data structures and how do I use them in my programs?

Goals of Course To understand implementation issues related to these data structures, and to see how they can be implemented in the Java programming language Data structures can be implemented in various ways, each of which has implications (ex: run-time differences, code complexity, modifiability) We should understand these data structures from an implementer's point of view: How can these data structures be effectively implemented?

Goals of Course To understand and utilize programming ideas and techniques utilized in data structure implementation Object-oriented programming, dynamic memory utilization, recursion and other principles must be understood in order to effectively implement data structures What tools must I know and be able to use in order to implement data structures?

Goals of Course To learn more of the Java programming language and its features, and to become more proficient at programming with it Java is a very large language with extensive capabilities As your programming skills improve, you can utilize more of these capabilities effectively Since I am working with Java, how well can I learn and use the language and its features?

Lecture 1: Getting Started
How and where can I use Java? Java is a platform-independent language, and thus can be used on many different platforms In CS 0401 Lab last term you used Windows Some of you may have Linux PCs or Macs Programs written in either place should run the same way Windows PC Java Program Mac

Lecture 1: Review Last term in CS 0401 you learned many things that you will need to use this term Basic Java program structure and syntax Java control structures and utilizing them effectively Loops, conditionals, etc Java methods and method calls Static methods vs instance methods Method overloading (ad hoc polymorphism) (Simple) recursive methods Java variables and objects Instance variables vs method variables Objects vs references Dynamic nature of Java objects

Interfaces, how and why to use them
Lecture 1: Review Java classes Syntax for writing new classes private, public and protected declarations Building new classes via composition and inheritance Polymorphism via method overriding and dynamic binding Interfaces, how and why to use them Simple Java files and graphics Exception handling If you are unsure about these things, look them over in your notes and in your CS 0401 textbook Also read Appendices A-E in the Carrano Data Structures text (some are online only) We will briefly go over Appendices C and D plus the Prelude and review some of these concepts in the first few lectures

Lecture 1: Classes and Objects
Classes are blueprints for our data The class structure provides a good way to encapsulate the data and operations of a new type together Instance data and instance methods Access restrictions (i.e. data hiding through private declarations) allow the implementation details of the data type to be hidden from a user public, protected and private allow various levels of accessibility

User of the class knows the nature of the data, and the public methods, but NOT the implementation details But does not need to know them in order to use the class Ex: BigInteger These are determined by the specifics of the private data and the method implementations We call this data abstraction This is related to abstract data types (ADTs), which we will discuss shortly

Java classes determine the structure of Java objects public declarations (typically methods) give the interface and functionality of the objects How the “outside world” communicates with the objects private declarations (typically data and some methods) hide the implementation details from the user To put it another way, Java objects are instances of Java classes

Class declaration keywords: public – accessible outside class private – inaccessible outside class protected – accessible only within class and subclasses [and same package] static – part of class rather than instance; shared by all instances final – constant – cannot be assigned (variables), overridden (methods) or subclassed (classes)

Lecture 2: References, Pointers and Memory
Other than the primitive type variables, all Java variables are references Draw output on board: 1) i is empty box, s is uninitialized pointer 2) 8463 in box, pointer to box containing “Hello There” 3) box for j with 8463, t points to same box as s, u points to new box 4) true, true, false Java references are similar to C++ pointers, BUT: 1) Dereferencing for instance variables (and methods) is automatic: C++: Java: foo * fooptr = new foo(); foo fooref = new foo(); (*fooptr).instancevar = 10; fooref.instancevar = 10; fooptr -> instancevar = 10; 2) Cannot access address value or use pointer arithmetic C++: Java: fooptr ++; Not allowed

Be careful when comparing Know when you want to compare references or contents For reference variables, we typically need to use a method to compare contents ex. for strings, equals() u.equals(s) returns true We can redefine equals() for our own classes as well Java does not allow operator overloading, so we cannot redefine comparison operators to compare contents – must use named methods

How do references and pointers relate? Many programming languages (ex: C, C++, Pascal) use pointer variables Pointers are variables that store addresses of other memory locations Pointers allow indirect access of the data in objects 101001 Object A X 010010 010010 Object B Y 101001

So the value stored in a pointer is an address However, if you dereference a pointer, you gain access to the object it "points to" X = Y; // Changes what X points to // X no longer has access to // Object B 101001 Object A X 101001 010010 Object B Y 101001

In C++, you dereference pointers using the * operator *X = *Y;// Changes contents of object // that X points to. The // value of X is unchanged 101001 Object A Copying the object, not the reference X 010010 010010 Object B Object A Y 101001

References in Java Behave in a way similar to pointers, but with more restriction Dereferencing is implicit – there is no dereference operator We cannot directly do what is shown in Slide 18 Accessing data / methods using the "dot" operator will implicitly dereference To copy an entire object we would need to use a method such as a clone() method or copy constructor Reference values (addresses) can be assigned but they cannot be manipulated No pointer arithmetic

But aliasing still occurs and you must be careful Be aware of when you want a new object or a reference to an old one StringBuilder S1 = new StringBuilder("Hello"); StringBuilder S2 = S1; S1.append(" There"); S2.append(" CS 0445 Students"); System.out.println(S1.toString()); Hello There CS 0445 Students Show on board

Java Memory Use All objects in Java are allocated dynamically Memory is allocated using the new operator Once allocated, objects exist for an indefinite period of time As long as there is an active reference to the object Objects that have no references to them are no longer accessible in the program Ex. Object B in slide 17 These objects are marked for GARBAGE COLLECTION

The Java garbage collector is a process that runs in the background during program execution When the amount of available memory runs low, the garbage collector reclaims objects that have been marked for collection A fairly sophisticated algorithm is used to determine which objects can be garbage collected If you take CS 1621 or CS 1622 you will likely discuss this algorithm in more detail If plenty of memory is available, there is a good chance that the garbage collector will never run See Example1.java

Lecture 2: Building New Classes
Java has many predefined classes Class library contains hundreds of classes, each designed for a specific purpose See API – However, in many situations we may need a class that is not already defined We will have to define it ourselves There are two primary techniques for doing this Composition (Aggregation) Inheritance

Lecture 2: Composition With composition, we build a new class using components (instance variables) that are from previously defined classes We compose the class from existent "pieces" "Has a" relationship between new class and old classes New class has no special access to its instance variable objects Methods in new class are often implemented by utilizing methods from the instance variable objects

Lecture 2: Composition public class CompoClass { private String name; private Integer size; public CompoClass(String n, int i) name = new String(n); size = new Integer(i); } public void setCharAt(int i, char c) StringBuilder b = new StringBuilder(name); b.setCharAt(i, c); name = b.toString(); We cannot access the inner representation of the String, and String objects are immutable, so we must change it in the rather convoluted way shown above

Lecture 2: Inheritance With inheritance, we build a new class (subclass) by extending a previously defined class (superclass) Subclass has all of the properties (data and methods) defined in the superclass "Is a" relationship between subclass and superclass Subclass is a superclass, and subclass objects can be assigned to superclass variables Not vice versa! Superclass IS NOT a subclass and superclass objects cannot be assigned to subclass variables

Lecture 2: Inheritance // Assume SubFoo is a subclass of Foo – see notes // below and on board Foo f1; SubFoo s1; f1 = new Foo(); // fine f1 = new SubFoo(); // also fine – however, now we // only have access to the public methods and // variables initially defined in class Foo() f1.foomethod(); // fine f1.subfoomethod(); // illegal ((SubFoo)f1).subfoomethod(); // fine, since now the ref. // has been cast to the actual class s1 = new SubFoo(); // also fine – now all SubFoo // public methods and variables are accessible s1.subfoomethod(); // fine s1.foomethod(); // also fine s1 = new Foo(); // illegal public class Foo { public void foomethod() { // implementation here } } public class SubFoo extends Foo public void subfoomethod()

Lecture 2: Polymorphism
Allows superclass and subclass objects to be accessed in a regular, consistent way Array or collection of superclass references can be used to access a mixture of superclass and subclass objects If a method is defined in both the superclass and subclass (with identical signatures), the version corresponding to each class will be used in a call from the array Idea is that the methods are similar in nature but the redefinition in the subclass gears the method more specifically to the data / properties of the subclass

Ex. Each subclass overrides the move() method in its own way Animal [] A = new Animal[3]; A[0] = new Bird(); A[1] = new Person(); A[2] = new Fish(); for (int i = 0; i < A.length; i++) A[i].move(); Animal move() move() Each call is syntactically identical Reference and method spec are the same Code executed is based on type of object being stored move()

Polymorphism is implemented utilizing two important ideas Method overriding A method defined in a superclass is redefined in a subclass with an identical method signature Since the signatures are identical, rather than overloading the method (ad hoc polymorphism), it is instead overriding the method For a subclass object, the definition in the subclass replaces the version in the superclass, even if a superclass reference is used to access the object Superclass version can still be accessed via the super reference

Dynamic (or late) binding The code executed for a method call is associated with the call during run-time The actual method executed is determined by the type of the object, not the type of the reference Polymorphism is very useful if we want to access collections of mixed data types consistently Ex: A collection of different graphical figures, each with a draw() method Each is drawn differently, so it has a different draw() method, but the call is consistent

Lecture 3: Abstract Classes
Sometimes in a class hierarchy, a class may be defined simply to give cohesion to its subclasses No objects of that class will ever be defined But instance data and methods will still be inherited by all subclasses This is an abstract class Keyword abstract used in declaration One or more methods may be declared to be abstract and are thus not implemented No objects may be instantiated

Lecture 3: Abstract Classes
Subclasses of an abstract class must implement all abstract methods, or they too must be declared to be abstract Advantages Can still use superclass reference to access all subclass objects in polymorphic way However, we need to declare the methods we will need in the superclass, even if they are abstract No need to specifically define common data and methods for each subclass - it is inherited Helps to organize class hierarchy See API for many examples

Java allows only single inheritance
Lecture 3: Interfaces Java allows only single inheritance A new class can be a subclass of only one parent (super) class There are several reasons for this, from both the implementation (i.e. how to do it in the compiler and interpreter) point of view and the programmer (i.e. how to use it effectively) point of view However, it is sometimes useful to be able to access an object through more than one type of superclass reference Interfaces allow us to do this

A Java interface is a named set of methods
Lecture 3: Interfaces A Java interface is a named set of methods Think of it as an abstract class with no instance data Static constants are allowed Default methods are allowed Static methods are allowed No instance data is allowed Regular instance methods have no bodies Interface itself cannot be instantiated Any Java class (no matter what its inheritance) can implement an interface by implementing the methods defined in it A given class can implement any number of interfaces

Lecture 3: Interfaces Ex: public interface Laughable { public void laugh(); } public interface Booable public void boo(); Any Java class can implement Laughable by implementing the method laugh() Any Java class can implement Booable by implementing the method boo()

Ex: Lecture 3: Interfaces
public class Comedian implements Laughable, Booable { // various methods here (constructor, etc.) public void laugh() System.out.println(“Ha ha ha”); } public void boo() System.out.println(“You stink!”);

Recall our previous discussion of polymorphism
Lecture 3: Interfaces Recall our previous discussion of polymorphism This behavior also applies to interfaces – the interface acts as a superclass and the implementing classes implement the actual methods however they want An interface variable can be used to reference any object that implements that interface However, only the interface methods are accessible through the interface reference Ex: Laughable [] funny = new Laughable[3]; funny[0] = new Comedian(); funny[1] = new SitCom(); // implements Laughable funny[2] = new Clown(); // implements Laughable for (int i = 0; i < funny.length; i++) funny[i].laugh(); See ex20.java, Laughable.java and Booable.java from CS 0401 Handouts

Lecture 3: “Generic” Operations
Let’s look at a simple example that should already be familiar to you: Sorting In CS 401 you should have discussed selection sort Simple algorithm: Find smallest, swap into location 0 Find next smallest, swap into location 1, etc. What if we want to sort different types (ints, doubles, Strings, or any Java type)? We need to write a different method for each one!!! The argument array must match the parameter array Or do we?? Can we write a single method that can sort anything? Use an interface! Discuss We can’t write a method that can sort “anything”, but we can write a method that can sort any Comparable objects – note the use of the interface here

Consider the Comparable<T> interface: It contains one method: int compareTo(T r); Returns a negative number if the current object is less than r, 0 if the current object equals r and a positive number if the current object is greater than r The parameter T allows arbitrary underlying data but with compile-time type checking – more on this later Consider what we need to know to sort data: is A[i] less than, equal to or greater than A[j] Thus, we can sort Comparable data without knowing anything else about it Awesome -- polymorphism allows this to work Note that this will not work for primitive types unless wrapper classes are used

Think of the objects we want to sort as “black boxes” We know we can compare them because they implement Comparable<T> Each type can implement compareTo() to be tailored to that type (works due to polymorphism) We don’t know (or need to know) anything else about them – even though they may have many other methods / instance variables Show on board Thus, a single sort method will work for an array of any class that implements Comparable<T> Did I mention that this was awesome!?

Lecture 3: "Generic" Operations
However – we do have some restrictions Cannot compare objects from inherently different types Don’t want to compare “apples to oranges” This is where the type parameter comes in Note that the argument to compareTo is type T T can be arbitrary but it must be compatible for two objects that are being compared If the types are not compatible, a compile-time error will be generated Prior to type parameters this check had to be done at run-time Generally speaking it is preferable to detect errors at compile-time if possible

Lecture 3: "Generic" Operations
For example, for 2 objects, C1 and C2, consider the call C1.compareTo(C2) If C1 is class T then C2 must be either class T or a subclass of T If C2 is any other type the program will generate a compilation error Let's put all of this together in another handout See Example2.java, People.java, Worker.java, Student.java, SortArray.java If you want to compare old vs. new generics, see also GenericDemo and modified versions of the classes

Lecture 3: More on Java Generics
Java allows for generic interfaces, classes and methods We saw interface example Comparable<T> Let's look at a simple class example and a simple method example We will revisit this topic again probably more than once Let's try to (very simply) mimic the functionality of a Java array We want to be a create an object with an arbitrary number of locations However, once created, the size is fixed

Lecture 3: More on Java Generics
We want the underlying type to be any Java type However, it should be homogeneous – cannot mix types (except if type is a superclass) We want to be able to assign a value to a location We want to be able to retrieve a value from a location We want to be able to tell the size of the array Let's see how to do this with Java Generics See MyArray.java and Example3.java Note that MyArray is not really a useful type – it is just meant to demonstrate parameterized Java types

Lecture 4: Abstract Data Types
Abstract Data Types (ADTs) We are familiar with data types in Java For example some primitive data types: int, float, double, boolean, or reference types such as String, StringBuilder, JButton and File We can think of these as a combination (or encapsulation) of two things: The data itself and its representation in memory For classes these are the instance variables The operations by which the data can be manipulated For classes these are the methods

For example, the BigInteger type in Java We can think of it simply as arbitrarily sized whole numbers, represented in some way in the computer, but this would be incomplete What makes integers useful is the operations that we can do on them, for example +, -, *, /, % and others It is understanding the nature of the data together with the operations that can be done on it that make ints useful to us For BigInteger the operators (ex: +, *) will not work, so we must implement the operations as methods

So where does the abstract part come in? Note that in order to use BigIntegers in our programs, we ONLY need to know what they are and what their operations do We do NOT need to know their implementation details Does it matter to me how the BigInteger is represented in memory? Does it matter how the actual division operation is done on the computer? For the purposes of using integers effectively in most situations…NO! These are abstracted out of our view!

More generally speaking, an ADT is a data type (data + operations) whose functionality is separated from its implementation The same functionality can result from different implementations Users of the ADT need only to know the functionality Naturally, however, to actually be used, ADTs must be implemented by someone at some point Implementer must be concerned with the implementation details In this course you will look at ADTs both from the user's and implementer's point of view

Lecture 4: ADTs vs. Classes
The previous slides should be familiar to you We have already discussed the idea of data abstraction from classes ADTs are language-independent representations of data types Can be used to specify a new data type that can then be implemented in various ways using different programming languages Classes are language-specific structures that allow implementation of ADTs Only exist in object-oriented or object-based languages

Lecture 4: ADTs vs. Classes
There is not a one-to-one mapping from ADTs to classes A given ADT can be implemented in different ways using different classes We will see some of these soon Ex: Stack, Queue, SortedList can be implemented in different ways A given class can in fact be used to represent more than one ADT The Java class ArrayList can be used to represent a Stack, Queue, Deque and other ADTs

Lecture 4: Interfaces as ADTs
Consider again interfaces Specify a set of methods, or, more generally a set of behaviors or abilities Do not specify how those methods are actually implemented Do not even specify the data upon which the methods depend These fit reasonably well with ADTs ADTs DO specify the data, but we can infer much about the data based on the methods within an interface

Lecture 4: Interfaces as ADTs
The text will typically use interfaces as ADTs and classes as ADT implementations Using the interface we will have to rely on descriptions for the data rather than actual data The data itself is left unspecified and will be detailed in the class(es) that implement the interfaces This is ok since the data is typically specific to an implementation anyway Ex: ADT Stack Push an object onto the top of the Stack Pop an object off the top of the Stack At this (ADT) level we don't care how the data is actually represented, as long as the methods work as specified

Lecture 4: ADTs for Collections of Data
Many ADTs (especially in this course) are used to represent collections of data Multiple objects organized and accessed in a particular way The organization and access is specified by the ADT Done through interfaces in Java The specific implementation of the data and operations can be done in various ways Done through classes in Java We will examine many of these this term!

Consider our first detailed ADT: the Bag
Lecture 4: ADT Bag Consider our first detailed ADT: the Bag Think of a real bag in which we can place things No rule about how many items to put in No rule about the order of the items No rule about duplicate items No rule about what type of items to put in However, we will make it homogeneous by requiring the items to be the same class or subclass of a specific Java type Let’s look at the interface See BagInterface.java

Note what is NOT in the interface:
Lecture 4: ADT Bag Note what is NOT in the interface: Any specification of the data for the collection We will leave this to the implementation The interface specifies the behaviors only However, the implementation is at least partially implied Must be some type of collection Any implementation of the methods Note that other things are not explicitly in the interface but maybe should be Ex: What the method should do Ex: How special cases should be handled We typically have to handle these via comments

Ex: public boolean add(T newEntry)
Lecture 4: ADT Bag Ex: public boolean add(T newEntry) We want to consider specifications from two points of view: What is the purpose / effect of the operation in the normal case? What unusual / erroneous situations can occur and how do we handle them? The first point can be handled via preconditions and postconditions Preconditions indicate what is assumed to be the state of the ADT prior to the method's execution Postconditions indicate what is the state of the ADT aftter the method's execution From the two we can infer the method's effect

Ex: for add(newEntry) we might have:
Lecture 4: ADT Bag Ex: for add(newEntry) we might have: Precondition: Bag is in a valid state containing N items Postconditions: Bag is in a valid state containing N+1 items newEntry is now contained in the Bag This is somewhat mathematical, so many ADTs also have operation descriptions explaining the operation in plainer terms More complex operations may also have more complex conditions However, pre and postconditions can be very important for verifying correctness of methods

The second point is often trickier to handle
Lecture 4: ADT Bag The second point is often trickier to handle Sometimes the unusual / erroneous circumstances are not obvious Often they can be handled in more than one way Ex: for add(newEntry) we might have Bag is not valid to begin with due to previous error newEntry is not a valid object Assuming we detect the problem, we could handle it by Doing a "no op" Returning a false boolean value Throwing an exception We need to make these clear to the user of the ADT so he/she knows what to expect

A Bag is a simple ADT, but it can still be useful
Lecture 4: Using a Bag A Bag is a simple ADT, but it can still be useful See examples in text Here is another simple one Generate some random integers and count how many of each number were generated There are many ways to do this, but one is with a bag See Example4.java Q: Is this the most efficient way of doing this? A: Hard to tell unless we can see how the Bag is implemented Let’s do that next!

Lecture 5: Implementing a Bag
Ok, now we need to look at a Bag from the implementer's point of view How to represent the data? Must somehow represent a collection of items (Objects) How to implement the operations? Clearly, the implementation of the operations will be closely related to the representation of the data Minimally the data representation will "suggest" ways of implementing the operations

Lecture 5: Array Implementation of a Bag
Let's first consider using an array Makes sense since it can store multiple values and allow them to be manipulated in various ways private Object [] bag; // old way private T [] bag; // current way Ok, but is just an array enough? We know the size of an array object, once created is fixed We also know that our Bag must be able to change in size (with adds and removes)

Lecture 5: Array Implementation of a Bag
Thus we need to create our array, then keep track of how many locations are used with some other variable private int numberOfEntries; Recall (from CS 0401) the following ideas: Physical size of the Bag Number of locations within the array Logical size of the Bag Number of items being stored in the bag The numberOfEntries variable is the maintaining the logical size

But how big to make the array (physical size)?
What if we run out of room? Note that the above questions are (mostly) irrelevant to the client, but are quite important to the implementer Two approaches to take: Use a fixed size and when it fills it fills Dynamically resize when necessary (transparently) You are doing this with your RandIndexQueue in Assignment 1

Lecture 5: Fixed Size Array
Idea: Initialize array in the constructor Physical size is passed in as a parameter (array length) Logical size maintained using numberOfEntries variable Once created, the physical size is constant as long as the list is being used Once array fills (i.e. logical size == physical size), any "add" operations will fail until space is freed (through "remove" or "clear") Advantage? Easier for programmer to implement

Disadvantages: ADT user (programmer) may greatly over-allocate the array, wasting space Overcompensates to not run out of room Program user (non-programmer) may run out of room at run-time If programmer does not do above Neither of these is desirable However, let's briefly look at this implementation anyway Much of it will be the same for the dynamic structure Only differences are when array fills

Let's start with the simple method we have been discussing: public boolean add (T newEntry) { // what do we need to do in the normal case? // what do we do in the abnormal case? } Recall our data: private T [] bag; private int numberOfEntries; Let's figure this out See board

Let’s look at code from text: public boolean add(T newEntry) { checkInitialization(); boolean result = true; if (isArrayFull()) result = false; } else { // assertion: result is true here bag[numberOfEntries] = newEntry; numberOfEntries++; } // end if return result; } // end add Note: isArrayFull() is a private method that is not part of the interface. It is used here specifically for the array implementation and is not part of the Bag ADT.

How about a bit more complicated operation? public boolean remove(T anEntry) {} What do we need to do here? Think of the “normal case” Must first find the item Then must remove it How? Think of unusual or special cases Let’s work up some code / pseudocode on the board

Consider the author’s code /** Removes one occurrence of a given entry from this bag. @param anEntry the entry to be removed @return true if the removal was successful, or false if not */ public boolean remove(T anEntry) { checkInitialization(); int index = getIndexOf(anEntry); T result = removeEntry(index); return anEntry.equals(result); } // end remove getIndexOf – simple sequential search (remember from CS 0401)? removeEntry – how to do this? Discuss Be careful with last line – note that this depends on how the equals method handles null arguments. See removeEntry().

private int getIndexOf(T anEntry) { int where = -1; boolean found = false; int index = 0; while (!found && (index < numberOfEntries)) if (anEntry.equals(bag[index])) found = true; where = index; } // end if index++; } // end while return where; } // end getIndexOf private method – not part of interface Loop condition: Continue as long as item is not found and not at end of logical array Test for found using equals method (rather than ==) If not found index will still be -1

private T removeEntry(int givenIndex) { T result = null; if (!isEmpty() && (givenIndex >= 0)) result = bag[givenIndex]; // entry to remove int lastIndex = numberOfEntries – 1; bag[givenIndex] = bag[lastIndex]; // replace entry to remove with last entry bag[lastIndex] = null; // remove reference to last entry numberOfEntries--; } // end if return result; } // end removeEntry Test to make sure index is valid Get entry to return [Do we have to do this? Discuss] Remove entry by copying last entry to its spot Implications of removing in this way? Discuss

Approach to implementing the other methods should be the same What is the method supposed to do? What can go wrong and what do we do about it? Does our code do what we want it to do? See text for discussion of more operations See ArrayBag.java for entire implementation Note: Due to publisher restrictions, I am putting the author’s implementations in a directory that is not accessible outside of Pitt’s domain If you want to access these you must do so from a Pitt IP address

Lecture 5: Dynamic Size Array
Idea: Array is created of some initial size Constructor can allow programmer to pass the size in, or we can choose some default initial size If this array becomes filled, we must: Create a new, bigger array Copy the data from the old array into the new one Assign the new array as our working array Some questions: How big to make the new array? How do we copy? What happens to the old array?

How big to make the new array? Clearly it must be bigger than the old array, but how much bigger? What must we consider when deciding the size? If we make the new array too small, we will have to resize often, causing a lot of overhead If we make the new array too large, we will be wasting a lot of memory Let's make the new array 2X the size of the old one This way we have a lot of new space but are not using outrageously more than we had before We will see more specifically why this was chosen later

How do we copy? This is pretty easy – just start at the beginning of the old array and copy index by index into the new array Not quite as simple for your RandIndexQueue - FYI Note that we are copying references, so the objects in the new array are the same objects that were in the old array What happens to the old array? It is garbage collected Let's try this on the board, then look at code See ResizableArrayBag.java and Example5.java Note how it is largely the same as ArrayBag.java Note: We could do 2) with our own code, but there is a standard method in the Arrays class that can do it for us: See:

Let's look in particular at the resizing process Resizing is initiated when an add is performed on a list with a full array: If array is not full this is identical to fixed array version The resizing process is transparent to the user of the ResizableArrayBag class For this operation, add() always succeeds public boolean add(T newEntry) { checkInitialization(); if (isArrayFull()) { doubleCapacity(); } bag[numberOfEntries] = newEntry; numberOfEntries++; return true; } // end add Will add() REALLY always succeed?

So what does doubleCapacity() do? Private method to do what we described: See Arrays.copyOf() API checkCapacity() makes sure size is not too big Note that instance variable numberOfEntries is not changed Why? private void doubleCapacity() { int newLength = 2 * bag.length; checkCapacity(newLength); bag = Arrays.copyOf(bag, 2 * bag.length); } // end ensureCapacity checkCapacity() does not really need to be here – without it the array can become arbitrarily large, which is the case for the standard Java data structures.

Lecture 6: Contiguous Memory Data Structures
Both Bag implementations so far use contiguous memory Locations are located next to each other in memory Given the address of the first location, we can find all of the others based on an offset from the first Benefits of contiguous memory: We have direct access to individual items Access of item A[i] can be done in a single operation 1 … i i+1 Show in more detail on board how addresses are calculated

Lecture 6: Contiguous Memory
Direct access allows us to use efficient algorithms such as Binary Search to find an item Arrays and array-based DS are also fairly simple and easy to use Drawbacks of contiguous memory Allocation of the memory must be done at once, in a large block as we just discussed If we allocate too much memory we are being wasteful If we do not allocate enough, we will run out We have seen how our Bag can resize transparently, but recall that this requires allocating new memory and copying into it, which takes time to do

Lecture 6: Contiguous Memory
Inserting or deleting data "at the middle" of an array may require shifting of the other elements Also requires some time to do We did not need to do this with our Bag, but other data structures (ex: Lists) may require this We will discuss the details of "how much" time is required later This deals with algorithm analysis

Lecture 6: Linked Data Structures
Let's concentrate on the drawbacks of contiguous memory Is there an alternative way of storing a collection of data that avoids these problems? What if we can allocate our memory in small, separate pieces, one for each item in the collection Now we allocate exactly as many pieces as we need Now we do not have to shift items, since all of the items are separate anyway Draw on board

But how do we keep track of all of the pieces? We let the pieces keep track of each other! Let each piece have 2 parts to it One part for the data it is storing One part to store the location of the next piece This is the idea behind a linked-list firstNode data data data data

Idea of Linked List: If we know where the beginning of the list is And each link knows where the next one is Then we can access all of the items in the list Our problems with contiguous memory now go away Allocation can be done one link at a time, for as many links as we need New links can be "linked up" anywhere in the list, without shifting needed Demonstrate on board

How can we implement linked lists?
Lecture 6: Linked Lists How can we implement linked lists? The key is how each link is implemented As we said, two parts are needed, one for data and one to store the location of the next link We can do this with a self-referential data type class Node { private T data; private Node next; … A NODE is a common name for a link in a linked-list Note why it is called "self-referential"

Lecture 6: Singly Linked Lists
Linked-List Implementation Variations Singly Linked List The simple linked-list we just discussed is a singly-linked list Links go in one direction only We can easily traverse the list from the front to the rear We CANNOT go backwards through the list at all This list is simple and (relatively) easy to implement, but has the limitations of any "one way street" This implementation is developed in Chapter 3 firstNode

Lecture 6: Singly Linked Lists
There are other variations of linked lists: Doubly linked list Circular linked list We will discuss these later For now we will keep things very simple

Lecture 6: Linked Bag Implementation
Let's look at this implementation a bit public class LinkedBag<T> implements BagInterface<T> { private Node firstNode; private int numberOfEntries; … private class Node private T data; private Node next; private Node(T dataPortion) { this(dataPortion, null); } private Node(T dataPortion, Node nextNode) { data = dataPortion; next = nextNode; } } // class Node } // class LinkedBag Note that Node is a private inner class

Lecture 6: Node As an Inner Class
Why is it done this way? Since Node is declared within LinkedBag, methods in LinkedBag can access private declarations within Node This is a way to get "around" the protection of the private data LinkedBag will be needing to access data and next of its Nodes in many of its methods We could write accessors and mutators within Node to allow this access However, it is simpler for the programmer if we can access data and next directly They are still private and cannot be accessed outside of LinkedBag On the downside, with this implementation, we cannot use Node outside of the LinkedBag class

Now let's see how we would implement some of our BagInterface methods public boolean add (T newEntry) { Node newNode = new Node(newEntry); // create Node newNode.next = firstNode; // link it to prev. front firstNode = newNode; // set front to new Node numberofEntries++; // increment entries return true; } // method add For the linked bag we add new values at the front, while we added new values at the end of the array. Since the Bag is not sorted, it does not logically matter. Compare to add() in the array implementation What is different? Is this a problem?

Trace on board Try a few adds in example Note insertion is at the front of the bag New node is created and newEntry is put in it New node becomes new front of list, push old front back Since BagInterface does not specify where to insert, we again do what is most convenient for our implementation Are there any special cases Ex: What if the bag is empty? firstNode will be null Will this be a problem? Any other special cases here?

Ok, that operation was simple How about something that requires a loop of some sort? Let’s look at the contains() method Just like for the array, we will use sequential search Just like for the array, we start at the beginning and proceed down the bag until we find the item or reach the end So what is different? How do we “move down” the bag? How do we know when we have reached the end? Discuss Let’s look at the code

public boolean contains(T anEntry) { boolean found = false; Node currentNode = firstNode; while (!found && (currentNode != null)) if (anEntry.equals(currentNode.data)) found = true; else currentNode = currentNode.next; } // end while return found; } // end getReferenceTo Loop will terminate when either found == true or null is reached (in which case found == false)

Let’s look at one more operation: public boolean remove(T anEntry) We want to remove an arbitrary item from the Bag How do we do this? Think about the contains() method that we just discussed How is remove similar and how is it different? Find the entry in question But now we need more than just a boolean result We must store a reference to the actual node so that we can do something with it Then remove it Must unlink the node from the list Let’s think about this

Consider again the properties of a Bag The data is in no particular order We could remove the actual Node in question but perhaps we can do it more easily The front Node is very easy to remove Trace on board So let’s copy the item in the front Node to the Node that we want to remove Then we remove the front Node Logically, we have removed the data we want to remove Keep in mind that the Nodes are not the data – they are simply a mechanism for accessing the data Also keep in mind that this would NOT be ok if the data needs to stay in some kind of order Note: The idea here is similar to what we did with the array implementation. However, there is an important difference. With the array, by moving an item into the empty spot rather than shifting, we are saving considerably on the runtime (going from linear runtime to constant runtime). For the linked implementation, the actual removal process is still only a few steps even in place, as we will see later with some other implementations. However, in either case, we are still required to find the object first, which takes linear time in an unordered collection such as a Bag.

private Node getReferenceTo(T anEntry) { boolean found = false; Node currentNode = firstNode; while (!found && (currentNode != null)) if (anEntry.equals(currentNode.data)) found = true; else currentNode = currentNode.next; } // end while return currentNode; } // end getReferenceTo Note that the logic here is the same as for contains What is different is what is returned

public boolean remove(T anEntry) { boolean result = false; Node nodeN = getReferenceTo(anEntry); if (nodeN != null) nodeN.data = firstNode.data; // copy data from firstNode = firstNode.next; // first Node and numberOfEntries--; // delete that Node result = true; // Operation succeeds } // end if return result; } // end remove Note that this returns boolean rather than the entry that is being removed

There are other methods that we have not discussed Look over them in the text and in the source code Look again at Example6.java Note (as we discussed) how the data is ordered differently in the different Bag implementations However, it is irrelevant to the functionality

Lecture 7: Node as a Separate Class
Node class as a separate (non-inner) class Some OO purists believe it is better to never "violate" the private nature of a class' data Making Node separate also allows it to be reused If done this way, the Node class must also be a parameterized type class Node<T> { private T data; // data portion private Node<T> next; // link to next node … And now in class LinkedList we would have private Node<T> firstNode;

Access to next and data fields must now be done via accessors and mutators, so these must be included in the Node<T> class Ex: getData(), getNextNode() accessors Ex: setData(), setNextNode() mutators Look at rest of Node<T> class code See handout Let's look at a method in LinkedBag.java we have already discussed, but now using this variation remove() method Differences from previous version are shown in red

public boolean remove(T anEntry) { boolean result = false; Node<T> nodeN = getReferenceTo(anEntry); if (nodeN != null) nodeN.setData(firstNode.getData()); firstNode = firstNode.getNextNode(); numberOfEntries-- result = true; } return result; Note that getReferenceTo() would also be different, using accessors and mutators rather than direct access to the nodes

Consider another ADT: the List
Lecture 7: ADT List Consider another ADT: the List We can define this in various ways – by its name alone it is perhaps only vaguely specified Let's look at how the text looks at it: Data: A collection of objects in a specific order and having the same data type The number of objects in the collection Operations: add(newEntry) add(newPosition, newEntry) remove(givenPosition)

See Ch. 12 for detailed specifications
Lecture 7: ADT List clear() replace(givenPosition, newEntry) getEntry(givenPosition) toArray() contains(anEntry) getLength() isEmpty() See Ch. 12 for detailed specifications Note that indexing for this ADT starts at 1, not 0 Odd but this is how the author defined it We will look at a few of these and see the similarities to and differences from our Bag ADT

Lecture 7: Using a List Recall that at this point we are looking at a List from a user's point of view So what can we use it for? A List is a very general and useful structure See ListInterface.java For example: We can use it for Last In First Out behavior (how?) We can use it for First in First Out behavior (how?) We can access the data by index and add/remove at a given location We can search for an item within the list

How about using it as a Bag?
Lecture 7: ADT List How about using it as a Bag? We could but would need to add the Bag methods It may not be the ideal ADT for some of these behaviors We will look at how some of these operations are done and their efficiencies soon However, we may choose to use it because it can do ALL of these things See Example7.java

Lecture 7: Java Standard List
Standard Java has a List interface Superset of the operations in author's ListInterface Some operations have different names Special cases may be handled differently Indexing starts at 0 But the idea is the same Look up List in the Java API See Example7b.java

Lecture 8: Implementing a List
Ok, now we need to look at a list from the implementer's point of view How to represent the data? Must somehow represent a collection of items (Objects) How to implement the operations? As discussed previously, the implementation of the operations will be closely related to the representation of the data Minimally the data representation will "suggest" ways of implementing the operations

Lecture 8: Linked List Implementation
Let’s first implement our ListInterface using a linked data structure [Note: The text uses array first but we are doing linked version first to aid with next assignment] Much of the implementation is identical to our LinkedBag Singly-linked list structure Node inner class with data and next fields Adding a new item at the front of the list is identical Finding an item in the list is identical However, there are some important differences between the two

The List interface requires data to be kept in positional order Thus, we cannot arbitrarily move data around Bag always removed Nodes from the front and moved data to allow arbitrary delete We can also insert and remove in a given position Will need to add and remove Nodes from the middle of the list This was not needed for LinkedBag Let’s focus on the parts of the LL that differ from the LinkedBag For example, consider the remove(int givenposition) method

public T remove(int givenPosition) What do we need to do here? We must first get to the object at givenPosition (i) There is a private method getNodeAt() to do this We will see the code soon Then we must "remove" it We must do this in such a way that the rest of the list is still connected We must link the previous node to the next node firstNode Previous Node Next Node i

But notice that by the time we find the node we want to delete, we have "passed" up the node we need to link Since the links are one way we can't go back Solution? Find the node BEFORE the one we want to remove Then get the one we want to remove and the one after that, and change the links appropriately firstNode nodeBefore nodeToRemove nodeAfter i

Let's look at the getNodeAt() method: Note that we start at the front of the list and follow the links down to the desired index Q: How does this compare to getting to a specific index in an array? /** Task: Returns a reference to the node at a given position. * Precondition: List is not empty; 1 <= givenPosition <= length. */ private Node getNodeAt(int givenPosition) { assert !isEmpty() && (1 <= givenPosition) && (givenPosition <= numberOfEntries); Node currentNode = firstNode; // traverse the list to locate the desired node for (int counter = 1; counter < givenPosition; counter++) currentNode = currentNode.getNextNode(); assert currentNode != null; return currentNode; } // end getNodeAt A: We can directly access any location in the array, but must traverse through the list to get to a location in the Linked List. This difference in access time for a specific location in a list is a significant difference in the array and linked implementations.

What if givenPosition > numberOfEntries? So why don't we handle this possible error? The method is private The idea is that as class designers, we make sure the error cannot occur – that is why it is an "assert" Users of the class cannot call this method, so there is no problem for them Ex: See public T getEntry(int givenPosition) The index test is done BEFORE getNodeAt() is called An assertion error This will crash our program!

Other issues? What else should we be concerned with when trying to delete a node? Are there any special cases we have to worry about? This is VERY IMPORTANT in many data structures and algorithms We discussed this for BagInterface but there were not really any problems But what about for ListInterface? if the index is invalid we cannot delete deleting the front node deleting the last remaining node (also the front node) Let's see, if the front node is deleted, the node before it will be ??????????? Special case!!!

Let's look at the code: public T remove(int givenPosition) { T result = null; // initialize return value if ((givenPosition >= 1) && (givenPosition <= numberOfEntries)) assert !isEmpty(); if (givenPosition == 1) // case 1: remove first entry result = firstNode.getData(); firstNode = firstNode.getNextNode(); } else // case 2: givenPosition > 1 Node nodeBefore = getNodeAt(givenPosition-1); Node nodeToRemove = nodeBefore.getNextNode(); result = nodeToRemove.getData(); Node nodeAfter = nodeToRemove.getNextNode(); nodeBefore.setNextNode(nodeAfter); } // end if numberOfEntries--; return result; // return result } // end if else throw new IndexOutOfBoundsException("Illegal Index"); } // end remove Special Case: First Node Normal Case Special Case: Invalid Index

Lecture 8: Singly Linked List Variations
First and Last References We discussed before that if we are inserting a node at the end of the list, we must traverse the entire list first to find the last previous node This is inefficient if we do a lot of adds to the end of the list [we'll discuss the particulars later] We could save time if we kept an additional instance variable (lastNode) that always refers to the end of the list Now adding to the end of the list is easy! However, it has some other interesting issues

See on board and discuss what the issues might be Thus, adding an extra instance variable to save time with one operation can increase the complexity of other operations Only by a small amount here, but we still need to consider it Let's look at an operation both without and with the lastNode reference Text looks at add() methods so let's look at a different one Let's try remove() Let's think about this

When, if at all, will we need to worry about the lastNode reference? With all of these methods we want to think about The "normal" case, or what we usually expect The "special" case that may only occur under certain circumstances Normal case: We remove a node from the "middle" of the list and the lastNode reference does not change at all Can we think of 2 special cases here? They are somewhat related

Removing the last node in the list This clearly will affect the lastNode reference How do we know when this case occurs? How do we handle it Removing the only node in the list Clearly this case is also 1) above, since the only node is also the last node However, we should consider it separately, since there may be special things that must be done if the list is becoming empty How do we handle it?

public T remove(int givenPosition) { T result = null; if ((givenPosition >= 1) && (givenPosition <= numberOfEntries)) assert !isEmpty(); if (givenPosition == 1) result = firstNode.getData(); firstNode = firstNode.getNextNode(); if (numberOfEntries == 1) lastNode = null; } else Node nodeBefore = getNodeAt(givenPosition-1); Node nodeToRemove = nodeBefore.getNextNode(); Node nodeAfter = nodeToRemove.getNextNode(); nodeBefore.setNextNode(nodeAfter); result = nodeToRemove.getData(); if (givenPosition == numberOfEntries) lastNode = nodeBefore; } // end if numberOfEntries--; } // end if else throw new IndexOutOfBoundsException("Illegal Index"); return result; } // end remove Code to handle deleting only node Code to handle deleting last node

Circular Linked List Now instead of null, the last node has a reference to the front node What is good about this? Which node(s) should we keep track of? Why? Think about adding at the beginning or end Can be effectively used for a Queue (see board) We will look at this more later lastNode

Lecture 8: Other Linked List Variations
Doubly Linked List Each node has a link to the one before and the one after Call them previous and next Now we can easily traverse the list in either direction Gives more general access and can be more useful This is more beneficial if we have a reference to the end of the list as well as the beginning, or we make it circular Used in standard JDK LinkedList and in author’s Deque Some operations may be somewhat faster But more overhead involved What overhead do we mean here? We may look in more detail if we have time

Lecture 9: Array Implementation of a List
Now consider using an array for our List Makes sense since it can store multiple values and allow them to be manipulated in various ways private T [] list; // same as for Bag We also need to keep track of the logical size private int numberOfEntries; To allow for an arbitrary number of items, we will dynamically resize when needed The same idea as for our Bag We will not spend too much time on this, since you are already familiar with much of this underlying impl.

Lecture 9: Array List Implementation
Let's start with an add method Unlike for Bag, with our List we can add at an arbitrary index public void add (int newPosition, T newEntry) { } Recall our data: private T [] list; private int numberOfEntries; Let's figure this out See board Note: Author will not use index 0 – the items will be in list[1] to list[numberOfEntries]

Let's look at the code from the text public void add(int newPosition, T newEntry) { checkInitialization(); // author does for all ADTS if ((newPosition >= 1) && (newPosition <= numberOfEntries + 1)) if (newPosition <= numberOfEntries) makeRoom(newPosition); list[newPosition] = newEntry; numberOfEntries++; ensureCapacity(); // look at this code also } else throw new IndexOutOfBoundsException(“Error”); } // end add Note that the List positions here go from 1 to length whereas the positions in the underlying array go from 0 to length-1, which can cause confusion in the implementation. In this case the author chooses to not use index 0 and thus make the array size one location larger than is needed. This problem does not exist in the standard Java List since the positions there are from 0 to length-1

How does makeRoom() work? A basic "shifting" algorithm However, be CAREFUL to shift from the correct side If you start on the wrong side you will copy, not shift Try going the other way and see the result! Show on board Note also that the method is private – why? private void makeRoom(int newPosition) { assert (newPosition >= 1) && (newPosition <= numberOfEntries+1); // move each entry to next higher index, starting at end of // list and continuing until the entry at newPosition is moved int newIndex = newPosition; int lastIndex = numberOfEntries; for (int index = lastIndex; index >= newIndex; index--) list[index+1] = list[index]; } // end makeRoom

What about removing data? public T remove (int givenPosition) { } Since the data must stay contiguous, in a sense we are doing the opposite of what we did to insert Remove and return the item Shift the remaining items over to fill in the gap Decrement numberOfEntries

Let's look at the code from the text public T remove(int givenPosition) { checkInitialization(); if ((givenPosition >= 1) && (givenPosition <= numberOfEntries)) { // get entry to be removed assert !isEmpty(); T result = list[givenPosition]; // move subsequent entries toward entry to be removed, // unless it is last in list if (givenPosition < numberOfEntries) removeGap(givenPosition); numberOfEntries--; return result; } // end if else throw new IndexOutOfBoundsException(“Error”); } // end remove Note that the List positions here go from 1 to length and index 0 is unused

How does removeGap() work? Again a basic "shifting" algorithm – now the other way We must still be careful about where to start Again try going the other way and see the result! Note that we did not need removeGap for the Bag, but we do for List – why? private void removeGap(int givenPosition) { assert(givenPosition >= 1) && (givenPosition < numberOfEntries); // shifts entries that are beyond the entry to be // removed to next lower position. int removedIndex = givenPosition; int lastIndex = numberOfEntries; for (int index = removedIndex; index < lastIndex; index++) list[index] = list[index+1]; } // end removeGap The List must maintain positional ordering

Special Cases? Invalid index must be handled but location of the add() or remove() does not really matter for the array Approach to implementing the other methods should be the same What is the method supposed to do? What can go wrong and what do we do about it? Does our code do what we want it to do? See text for discussion of more operations See AList.java for entire implementation Note: As with other author’s code segments, I will put this in a “Pitt only” directory

Lecture 9: Standard Java List Classes
We mentioned previously that in standard Java there is a List interface similar to the author's ListInterface So how is the standard List implemented? Recall that for now we are considering only array-based implementations ArrayList is a class developed as part of the standard Java Collections Framework Built from scratch to implement the List interface Uses a dynamic expanding array (similar to what we discussed but with a slightly different size increase factor)

In real applications where a List is needed you will likely use ArrayList Vector is a class created before the Java Collections Framework was developed Designed to be a dynamically expanding collection When the Collections Framework was developed, Vector was retrofitted into it through the addition of the standard List methods Previous methods were also kept, so for a lot of operations there are two (almost) equivalent methods in the Vector class, for ex: public E remove(int index) public void removeElementAt(int index) Note return types

There is one other interesting difference between Vector and ArrayList Vector is synchronized and ArrayList is not What does this mean? If multiple Threads attempt to modify a Vector "at the same time", only one will be allowed to do so Idea is that the data remains consistent when used with multiple Threads ArrayList makes no such guarantee So what are Threads, you ask? Objects that allow parts of programs to execute in "pseudo-parallel" We will not really discuss them here You may discuss them in another course

Lecture 9: Algorithm Analysis
Consider different ADT implementations We have talked about efficiency differences, but we have been somewhat vague about it Now we will look at algorithm efficiencies in a more formal way Mathematically Why do we care about formalizing this? Consider all of the work involved in implementing a new ADT It is non-trivial to get all of the operations working correctly Many special cases and much debugging is required

If we could determine whether or not an implementation was good before actually doing the work, it could save us a lot of time Inefficient potential implementations could be abandoned before they are even done Ex: Sum of integers example in text (Sections ) Ex: One you should be familiar with Searching a sorted array Assume the array has N items in it Sequential search can take up to N tests to find the item Binary search will take at most log2N tests to find the item So is this a big difference?

Let's first look at the tests for 1 search: N lg2N 8 3 16 4 32 5 64 6 … 1024 10 1M 20 1G 30

Now consider multiple searches Let's say for example I need to do 1 million searches of 1 million items For sequential search this could be up to 1M x 1M = 1T = 1012  WOW! For binary search this would be 1M x 20 = 20M = 2x107  What a difference Assume each test takes a nanosecond (10-9) For sequential search we need 1012(10-9) = 103 seconds = (103)(1 minute/60 seconds) = minutes

For binary search we need 2x107(10-9) = 0.02 sec The difference is amazing Just rethinking our algorithm takes us from something that would take minutes to something that just takes a fraction of a second Other examples can have even more extreme differences See CS 1501 By analyzing our algorithm BEFORE implementing it, we can thus avoid implementing algorithms that will require too much time to run A little analysis saves us a lot of programming!

Lecture 10: Algorithms and Complexity
Measuring Execution Time How to compare execution times of algorithms? Certainly we can time them empirically This will give us actual run-times that we can use to compare Very useful for algorithms/ADTs that have already been developed into programs – already implemented But we said previously that often it is good to get a ballpark on the runtime of an algorithm/ADT BEFORE actually implementing it Perhaps we wouldn't want to go through the effort if the algorithm is not going to be useful See example on board

Asymptotic analysis Do not time actual program – in fact we may not necessarily even have a program Instead do the following: Determine some key instruction or group of instructions that controls the overall run-time behavior of the algorithm For example, for sorting we need to compare items to each other Even though sorting involves other instructions, we can say that the overall run-time is directly proportional to the number of comparisons done

Determine a formula / function for how the number of key instructions increase as the problem size increases (typically we use the variable N for the problem size) We typically are concerned with two different results Worst Case Time: What is the formula for the MAXIMUM number of key instructions relative to N We should know what the worst case time can be so that we can plan for it if necessary Average Case Time: What is the formula for the AVERAGE number of key instructions relative to N How will the algorithm do normally?

Only worry about the order of magnitude We use the measure Big-O for this For a given formula, we ignore lower order terms and constant multipliers Ex: Let's say we determine the formula for the comparisons for a given sorting algorithm in the worst case to be F(N) = (N2/2) – (N/2) We say the Big-O run-time of this sorting algorithm is O(N2) We ignore lower order terms because … they become less significant as the problem size increases Compare some function growth rates to see this point – see board.

We ignore constant multipliers because … they can depend on programmer, lang., computer, etc. Program A written by Joe Schmoe runs in time 4N Program B written by Jill Schmill runs in time 2N Maybe Jill is a better programmer than Joe Maybe one compiler makes more efficient code than the other How about some simple examples: Constant time O(1) Y = X; Z = X + Y; Linear time O(N) for (int i = 0; i < N; i++) do_some_constant_time_operation Why is Z = X + Y constant time? Discuss.

Quadratic Time O(N2) for (int i = 0; i < N; i++) for (int j = 0; j < N; j++) do_some_constant_time_op; We will look at some others as well later on So for searching that we mentioned previously: Key instruction is a comparison Sequential Search is O(N) Why? Single while loop with up to N iterations / comps Binary Search is O(lg2N) Why? This one is a bit trickier We still have a loop, but now the number of iterations is very different Let's look at the code from the standard Java library in the java.util.Arrays class

What is the "worst case" for this? public static int binarySearch(Object[] a, Object key) { int low = 0; int high = a.length-1; while (low <= high) { int mid =(low + high)/2; Object midVal = a[mid]; int cmp = ((Comparable)midVal).compareTo(key); if (cmp < 0) low = mid + 1; else if (cmp > 0) high = mid - 1; else return mid; // key found } return -(low + 1); // key not found.

To simplify calculations we'll cheat a bit: Assume that the array is cut exactly in half with each iteration In reality it may vary by one element either way Assume that the initial size of the array, N is an exact power of 2, or 2K for some K In reality it can be any value However, the assumption will not affect our results Ok, so we have the following: Initially, at iteration 0, N0 = 2K At iteration 1, N1 = N0/2 = (in terms of K) ... At iteration K, NK = 1 = (in terms of K) 2K-1 20

We do one comparison (test) per iteration Thus we have a total of K+1 comparisons maximum But N = 2K So K = Which makes K + 1 = This leads to our final answer of Note: Our simplifying assumptions do not change our final answer: May change the result by an iteration or two but the Big-O will still be the same Discuss Yes there is mathiness here! But the highest mathicity you will see is yet to come! lg2N lg2N + 1 O(lg2N)

Let's look at another example Consider our Bag implementations We can now formally analyze the run-times of some operations, to determine which implementation is better for which operations (if at all)

add(newEntry) Recall that this version of the method adds to the end of the array or the front of the linked list Runtime for Resizable Array ? What about the time to resize? O(1)? We can go directly to the last location and insert there The answer above is a bit deceptive Some adds take significantly more time, since we have to first allocate a new array and copy all of the data into it – O(N) time So we have O(N) + O(1)  O(N) total

So we have an operation that sometimes takes O(1) and sometimes takes O(N) How do we handle this issue? Amortized Time (see ) Average time required over a sequence of operations Individual operations may vary in their run-time, but we can get a consistent time for the overall sequence Let's stick with the add() method for Resizable Array Bag and consider 2 different options for resizing: Increase the array size by 1 each time we resize Double the array size each time we resize (which is the way the authors actually did it)

Remember This!! Lecture 10: Algorithms and Complexity Increase the array size by 1 each time we resize Note that with this approach, once we resize we will have to do it with every add Thus rather than O(1) our add() is now O(N) all the time Specifically, assume the initial array size is 1 On insert 1 we just add the item (1 assignment) On insert 2 we allocate and assign 2 items On insert 3 we allocate and assign 3 items … On insert i we allocate and assign i items Overall for N add() ops look at the total number of assignments we have to make: … + N = This is an arithmetic sum. From Discrete Math we know that the sum of integers from 1 to N = N(N+1)/2. For a very simple proof of this list the sequence from 1 to N in both forward and reverse order, then add the individual items. You see the term (N+1) is listed N times. But this is 2 times the result we need so we divide by 2 to get the answer. Remember this sum and its solution – you will see it OFTEN! N(N+1)/2  O(N2)

Note that this O(N2) time is for the sequence of N operations The amortized time for one operation is thus O(N2)/N = O(N) This is linear and thus makes the overall add() operation for the ArrayBag linear We would like to do better than this

Double the array size each time we resize Insert # # of assignments End array size 1 2 2 = 3 3 = 4 5 5 = 8 … 9 9 = 16 17 17 = 32 Blue is cost of actually adding the item Red is cost of copying data to a new array

Note that every row has 1 assignment (blue) Rows that are 2K + 1 for some K have an additional 2K assignments (red) to copy data So for N adds, we have a total of N assignments for the actual add … + 2x for the copying What is x? This gives us the geometric series [ceiling (lg2N) – 1] ( = lg2N – 1 if N is a power of 2) Generally, for a geometric series with base 2 we have: Sum (from 0 to k) 2i = 2k+1 – 1 In the case above k is lg2N – 1 x is the greatest integer such that 2^x < N

Total is N + (N-1) = 2N-1  O(N) Since we did N add() operations overall, our amortized time is O(N)/N = O(1) – constant Recall that when increasing the array by 1 we had O(N2) overall for the sequence, which gives us O(N) in amortized time Note how much better our performance is when we double the array size Ok, that one was a bit complicated Had a good deal of math in it But that is what algorithm analysis is all about If you can do some math you can save yourself some programming!

What about the run-time for the LinkedBag? Recall the add() method that adds to the front of the list Discuss Text discusses other Bag operations It turns out that for the Bag, the run-times for the array and the linked list are the same for every operation This is not always be the case Consider, for example the List ADT

Lecture 11: List Run-time Complexity
What are the Big-O complexities for our List implementations? What is different about the List from the Bag that may have an impact on the run-times? Let’s look at one operation in particular to highlight the difference: getEntry(int i) This accesses an arbitrary location in the list Not part of the Bag See Example8.java Let’s compare our AList and LList implementations with regard to this operation

For the AList, we simply index our array, and can access entry in: O(1) time in every case What about our LList? Now it depends on the index Sequential access requires us to traverse the list i Nodes to get to Nodei Worst case? getEntry() is O(N) worst case for the LinkedList Note that it could be less, depending on where the object is located So maybe we should also consider the average case here, to be thorough

To do this we need to make an assumption about the index chosen Let's assume that all index values are equally likely If this is not the case, we can still do the analysis, if we know the actual probability distribution for the index choice Our assumption means that, given N choices for an index, the probability of choosing a given index, i, (which we will call P(i)) is 1/N for any i Let's define our key operation to be "looking at" a node in the list So for a given index i, we will require i operations to get to that location Let's call this value Ops(i)

Now define the average number of operations to be: Ave Ops = Sum_over_i (Ops(i) * P(i)) = Sum_over_i (i * 1/N) = 1/N * Sum_over_i (i) = 1/N * [N * (N+1)]/2 = (N+1)/2 In an absolute sense, this is better than the worst case, but asymptotically it is the same (why?) So in this case the worst and average cases for getEntry() on a linked list are the same  O(N) So we see: getEntry() for array: O(1) getEntry() for linked list: O(N) Think about this in terms of walking down a line of people looking for someone in particular. If the operation is "looking at someone" then the number of operations required to get to person i is equal to i. Now consider the case where you want to consider how long on average it takes to look (separately) for everyone in the line. This would be 1 operation for the first person plus 2 operations for the 2nd person, etc, divided by the number of people in the list.

Lecture 11: List Run-Time Complexity
We will see later how Iterators allow us improve this access for linked lists Note also that this result does NOT mean that the AList is always better than the LList add(1) and remove(1) for the AList are O(N) add(1) and remove(1) for the LList are O(1) However, note that in the worst case add(i) and remove(i) are O(N) for both the AList and LList For different reasons -- discuss To evaluate we need to assess which operations we need and how often we need them

Lecture 11: Queue Complexity
What about the Queue implementations? We have seen 3: offer() at front, poll() at end offer() is O(N) due to shifting poll() is O(1) offer() at end, poll() from front offer() is O(1) poll() is O(N) due to shifting You did these in Recitation Exercise 1 In both cases one of the operations is poor Let’s assume that everything we put into the queue we eventually take out, and consider the amortized time per op for N offers followed by N polls

In case 1) offer() is … + N = N(N+1)/2  O(N2) Discuss why and poll() is … + 1 = N  O(N) Total is O(N2) + O(N) = O(N2) This leads to O(N2)/N = O(N) amortized per op In case 2) we have the opposite (do the math as an exercise) The total is now O(N) + O(N2) = O(N2) And has the same amortized time of O(N) per op

Use two indexes to move down the array in a circular fashion for both offer() and poll() You did this in Assignment 1 In this case offer() and poll() are both O(1) [assuming efficient resizing as discussed previously] Other Queue implementations can also achieve an O(1) run-time Circular linked list (singly linked or doubly linked could both work here) Linked list with front and rear pointer

Recursion Idea Requirements
Lecture 11: Recursion Recursion Idea Some problem P is defined/solved in terms of one or more problems P', which are identical in nature to P but smaller in size Requirements 1 or more base cases in which no recursive call is made 1 or more recursive cases in which the algorithm is defined in terms of itself The recursive cases must eventually lead to a base case

Simple Examples of Recursive Algorithms
Lecture 11: Recursion Simple Examples of Recursive Algorithms A lot of recursive problems have their origins in mathematics Factorial – N! = Iterative definition: N * (N-1) * (N-2) * … * 1 Recursive definition: N! = N * (N-1)! when N > 0 N! = 1 when N = 0 Let's look at our 3 requirements: 1 base case when N = 0 1 recursive case when N > 0 Since recursive case has argument of N-1, it should always lead to a base case…but does it always? Be CAREFUL – make sure it always works! Trace a simple example on the board

Let's look at another simple example
Lecture 11: Recursion Let's look at another simple example Integer Powers – XN Iterative Definition: X * X * X * … * X – N times Recursive Definition: XN = X * X(N-1) when N > 0 XN = 1 when N = 0 Our 3 requirements 1 base case when N = 0 1 recursive case when N > 0 Decrementing of N gives similar situation to that of factorial Normally base case is always reached, unless N is initially negative

How to code using recursion?
Lecture 11: Recursion How to code using recursion? Many recursive programs are very similar to the underlying mathematical definitions Let's look at Factorial: public long factorial (int N) { if (N < 0) throw new IllegalArgumentException(); if (N <= 1) return 1; return N * factorial(N-1); } Note that negative N generates an exception Function is calling itself, using the result in the return expression

How does recursion work?
Lecture 11: Recursion How does recursion work? 2 important ideas allow recursion to work Activation Record (AR) A block of memory allocated to store parameters, local variables and the return address during a function/method call An AR is associated with each method CALL, so if a method is called multiple times, multiple ARs are created Run-Time Stack (RTS) Area of computer's memory which maintains ARs in Last In First Out (LIFO) order Stack idea is intuitive – we will discuss formally soon

When a method is called See Example9.java
Lecture 12: Recursion When a method is called An AR containing the parameters, return address and local variables is pushed onto the top of the RTS If the method subsequently calls itself, a new, distinct AR containing new data is pushed onto the top of the RTS The AR at the TOP of the RTS represents the currently executing call ARs below represent previous calls that are waiting to be returned to When top call terminates, control returns to the address from top AR and then the top AR is popped from the RTS See Example9.java Trace factorial using run-time stack

Let's look at one more simple example
Lecture 12: Recursion Let's look at one more simple example Sequential Search Find key in an array by checking each item in sequence We know how to do this iteratively Simple for loop or while loop to go through each item We have done this with the contains() method for both the dynamic array bag and the linked bag Let's see how to do it recursively Remember to always consider the problem in terms of a smaller problem of the same type Remember that we need Base case Recursive case Recursive calls must lead to base case Show this on the board

Lecture 12: Recursion In order to search for a key in an array of length N we check the length If length == 0, we are done (base case not found) Q: How is "length" changed between calls? A: Logical length is checked and updated via indices This is commonly done with recursive methods on arrays Else check the first element of the array If first element == key, we are done (base case found) Else Sequential Search the remaining N-1 elements (recursive case) Once we have this idea, we can quickly convert it into code See SeqSDemo.java In recursive algorithms on arrays, we will usually update the "length" of the array logically by changing beginning and/or ending index values. To actually reduce the size of the physical array would require allocating a new, smaller array which would add an incredible amount of overhead to the recursive process.

Let’s also consider recursive sequential search of a linked list
Lecture 12: Recursion Let’s also consider recursive sequential search of a linked list What is similar and what is different from the array based version? Discuss Base case not found? Base case found? Recursive case? Be careful with special cases! Consider base case of list.getNextNode() == null What if list is initially empty? See SeqSLinked.java

Lecture 12: Recursion and Divide and Conquer
So far Recursive algorithms that we have seen (see text for more) are simple, and probably would NOT be done recursively The iterative solutions work fine They are just used to demonstrate how recursion works However, recursion often suggests approaches to problem solving that are more logical and easier than without it For example, divide and conquer

Let's look at one of our earlier recursive problems – Power function, XN We have already seen a simple iterative solution using a for loop We have already seen and discussed a simple recursive solution Note that the recursive solution does recursive calls rather than loop iterations However both algorithms have the same runtime: We must do O(N) multiplications to complete the problem Can we come up with a solution that is better in terms of runtime? Let's try Divide and Conquer

The idea is that a problem can be solved by breaking it down to one or more "smaller" problems in a systematic way Usually the subproblem(s) are a fraction of the size of the original problem Note how this differs from the more general recursive definition where they must be simply smaller Usually the subproblems(s) are identical in nature to the original problem It is fairly clear why these algorithms can typically be solved quite nicely using recursion

We can think of each lower level as solving the same problem as the level above The only difference in each level is the size of the problem, which is ½ of that of the level above it Note how quickly the problem size is reduced We have seen this already with Binary Search

Lecture 13: Exam One Exam One Material up to and including Lecture 12

Lecture 14: Divide and Conquer Power Function
How can we apply this to the Power fn? We typically need to consider two important things: How do we break up or "divide" the problem into subproblems? In other words, what do we do to the data to process it before making our recursive call(s)? How do we use the solutions of the subproblems to generate the solution of the original problem? In other words, after the recursive calls complete, what do we do with the results? You can also think of this is “how do we put the pieces back together?” For XN the problem "size" is the exponent, N So a subproblem would be the same problem with a smaller N

Let's try cutting N in half – use N/2 1) We want to define XN somehow in terms of XN/2 We can't forget the base case 2) We need to determine how the original problem is solved in terms of the solution XN/2 Do on board (and see notes below) Will this be an improvement over the other 2 versions of the function? Problem size is being cut in half each time Informal analysis shows we only need O(log2N) multiplications in this case (see text ) Same idea as the analysis for binary search We know this is a big improvement over O(N) Let's look at the code – Power.java XN = (XN/2)2 Ok, but we need a base case as always We can use the same as before: X0 = 1 But will this always work? We need to make sure that our solution is always correct. Let's try 23 = (21)2 = ((20)2)2 = (12)2 = 1 What went wrong? Well the solution above works only when the exponent is EVEN. If the exponent is odd, due to integer division truncation, we lose information. We need a different case here. Let's try again XN = (XN/2)2 when N is even and > 0 = X * (XN/2)2 when N is odd and > 0 = 1 when N = 0

Caution: Recursive calls spawn other recursive calls It is VERY important to understand how these calls proceed when analyzing / utilizing recursive algorithms Sometimes a correct algorithm is not necessarily a desirable one, even if it “seems” good Consider XN as we have just discussed Let’s look at the problem expressed in two different, but mathematically equivalent ways

Solution we just discussed: XN = (XN/2) N even, > 0 XN = X*(XN/2)2 N odd, > 0 XN = N = 0 Analysis for this is similar to that of Binary Search Run-time stack grows to height of ~lgN At each level we do one or two multiplications One for the square (always) One for odd N (sometimes) Alternate solution XN = (XN/2)(XN/2) N even, > 0 XN = X*(XN/2)(XN/2) N odd, > 0 XN = N = 0 Mathematically you can argue that these are the same From a programming point of view they are NOT This one is TERRIBLE Why? Discuss Look at execution trace This one does O(N) multiplications Same as non-divide and conquer See Power2.java Trace is a tree with height lgN. Even though this is the same height as the first solution, in the first solution we have a linear chain of calls, with ~lgN total calls. In this case our tree is a full binary tree and has a total of ~N calls in it. We will talk more about binary trees soon.

One final caution about this particular problem: In the prev analysis we calculated “number of mults”, assuming (more or less) that a mult can be done in O(1) This is not true – in fact is a function of the number of bits in X (actual time discussed in CS 1501). If X is an int or long then we can think of this as constant If X is arbitrarily large then the run-time for the multiplication will also grow quickly Also, even storing / allocating space for the numbers will be time consuming (think: a square “doubles” the bits) For very large numbers a powerMod() method may be more useful: XN mod Z for some Z This restricts the maximum size of the result Will discuss in CS 1501

Lecture 14: Recursion and Binary Search
Now let's reconsider binary search, this time using using recursion Recall that the data must be in order We are searching for some object S How do we divide? Cut the array in half – makes sense since the iterative version cuts the array in half as well How do we use the subproblem results to solve the original problem? This is interesting – in fact we may not really need to do anything here at all – let's see Be careful not to make unnecessary calls – this eliminates the advantage of Divide and Conquer. See Power2.java

Ok, what about base case? Two cases actually Same idea as sequential search Base case not found – logical array size is down to zero Base case found – key matches current item What about the recursive case? Consider the middle element, M, and check if S is: Equal to M: you are done and you have found it One of the base cases Less than M: recurse to the left side of the array Greater than M: recurse to the right side of the array Same logic as the iterative version

Proceeding in this fashion removes ½ of the remaining items from consideration with each guess i.e. with each recursive call Let's compare this to iterative binary search We will also compare it to sequential search See BSTest.java Counts the number of comparisons required for the searches Clearly as N gets larger the difference becomes quite significant Also read Chapter 18 of the Carrano text It discusses both sequential search and binary search

Lecture 14: More Recursion
So far Every recursive algorithm we have seen can be done easily in an iterative way Even the divide and conquer algorithms (Binary Search, Power function) have simple iterative solutions Can we tell if a recursive algorithm can be easily done in an iterative way? Yes – any recursive algorithm that is exclusively tail recursive can be done simply using iteration without recursion Most algorithms we have seen so far are exclusively tail recursive

Lecture 14: Tail Recursion
So what is tail recursion? Recursive algorithm in which the recursive call is the LAST statement in a call of the method Look at algorithms so far to see this is true (ignore trace versions, which add extra statements) Note Power does some math after the call, but it can still be done easily in an iterative way, even the divide and conquer version What are the implications of tail recursion? Any tail recursive algorithm can be converted into an iterative algorithm in a methodical way In fact some compilers do this automatically

Lecture 14: Overhead of Recursion
Why do we care? Recursive algorithms have overhead associated with them Space: each activation record (AR) takes up memory in the run-time stack (RTS) If too many calls "stack up" memory can be a problem We saw this when we had to increase the stack size for BSTest.java Time: generating ARs and manipulating the RTS takes time A recursive algorithm will always run more slowly than an equivalent iterative version

Lecture 15: Overhead of Recursion
So what good is recursion? For some problems, a recursive approach is more natural and simpler to understand than an iterative approach Once the algorithm is developed, if it is tail recursive, we can always convert it into a faster iterative version (ex: binary search, power) For some problems, it is very difficult to even conceive an iterative approach, especially if multiple recursive calls are required in the recursive solution Example: Backtracking problems

Lecture 15: Recursion and Backtracking
Idea of backtracking: Proceed forward to a solution until it becomes apparent that no solution can be achieved along the current path At that point UNDO the solution (backtrack) to a point where we can again proceed forward Example: Traversing a maze Example: 8 Queens Problem How can I place 8 queens on a chessboard such that no queen can take any other in the next move? Recall that queens can move horizontally, vertically or diagonally for multiple spaces See on board

Lecture 15: 8 Queens Problem
How can we solve this with recursion and backtracking? We note that all queens must be in different rows and different columns, so each row and each column must have exactly one queen when we are finished Complicating it a bit is the fact that queens can move diagonally So, thinking recursively, we see the following To place 8 queens on the board we need to Place a queen in a legal (row, column) Recursively place 7 queens on the rest of the board Where does backtracking come in? Our initial choices may not lead to a solution – we need a way to undo a choice and try another one See example on board

Using this approach we come up with the solution as shown in 8-Queens handout JRQueens.java Idea of solution: Each recursive call attempts to place a queen in a specific column A loop is used, since there are 8 squares in the column For a given call, the state of the board from previous placements is known (i.e. where are the other queens?) This is used to determine if a square is legal or not If a placement within the column does not lead to a solution, the queen is removed and moved "down" the column

When all rows in a column have been tried, the call terminates and backtracks to the previous call (in the previous column) If a queen cannot be placed into column i, do not even try to place one onto column i+1 – rather, backtrack to column i-1 and move the queen that had been placed there See handout for code details Why is this difficult to do iteratively? We need to store a lot of state information as we try (and un-try) many locations on the board For each column so far, where has a queen been placed?

The run-time stack does this automatically for us via activation records col (parameter) and row (local variable) are separate for each call of the method Without recursion, we would need to store / update this information ourselves This can be done (using our own Stack rather than the run-time stack), but since the mechanism is already built into recursive programming, why not utilize it? There are many other famous backtracking problems

Lecture 16: More Backtracking
Let’s look at one more backtracking example Finding words in a 2-d grid of letters We are given a grid and a word Is the word located somewhere within the grid? Each letter must touch the previous letter But we can move right / down / left / up Think recursively! Trace the algorithm Try to imagine how to do this without recursion…very difficult! See FindWord.java

Lecture 16: Towers of Hanoi
Another Famous Recursive Algorithm: Towers of Hanoi Problem Problem: We have 3 towers On first tower we have disks of decreasing size Goal is to get all disks onto last tower, but We can only move one disk at a time We can never put a larger disk on top of a smaller one Let's play and see why it is so difficult to solve in an iterative way Volunteer? Go over JRHanoi.java. Try interactive part. Run demonstration. Trace algorithm

Lecture 16: Towers of Hanoi
Why is this problem difficult iteratively? A recursive algorithm with a single recursive call still provides a linear chain of calls Calls build run-time stack Stack shrinks as calls finish

Lecture 16: Execution Trees
When a recursive algorithm has 2 calls, the execution trace is now a binary tree, as we saw with the trace on the board This is execution is more difficult to do without recursion To do it, programmer must create and maintain his/her own stack to keep all of the various data values This increases the likelihood of errors / bugs in the code Soon we will see some other classic recursive algorithms with multiple calls Ex: MergeSort, QuickSort Note: We will discuss binary trees in detail later

Sorting is a very common and useful process
Lecture 17: Sorting Sorting is a very common and useful process We sort names, salaries, movie grosses, Nielsen ratings, home runs, populations, book sales, to name a few It is important to understand how sorting works and how it can be done efficiently By default, we will consider sorting in increasing order: For all indices, i, j: if i < j, then A[i] <= A[j] Note we are allowing for duplicates here Note that for decreasing order we simply change right side to A[i] >= A[j]

Simple Sorting Algorithms
Lecture 17: Simple Sorts Simple Sorting Algorithms Insertion Sort Idea: "Remove" the items one at a time from the original array and "Insert" them into a new array, putting them into the correct relative sorted order as you insert We could accomplish this by using two arrays as implied above, but that would double our memory requirements We'd rather be able to sort in place Use only a constant amount of extra memory

Lecture 17: Simple Sorts To actually implement we are going to think of the array or in two parts In each iteration of our outer loop, we will take an item out of the UNSORTED section and put it into its correct relative location in the SORTED section SORTED UNSORTED 1 2 3 4 5 6 7 20 40 70 30 50 10 80 60 Go over the process showing how each row results from next iteration of outer loop Note that from one row to next in above table requires multiple operations

Let's look at some code (from prev. text edition)
Lecture 17: Simple Sorts Let's look at some code (from prev. text edition) public static <T extends Comparable<? super T>> void insertionSort(T[] a, int n) { insertionSort(a, 0, n - 1); } // end insertionSort public static <T extends Comparable<? super T>> void insertionSort(T[] a, int first, int last) { int unsorted, index; for (unsorted = first + 1; unsorted <= last; unsorted++) { // Assertion: a[first] <= a[first + 1] <= ... <= a[unsorted - 1] T firstUnsorted = a[unsorted]; insertInOrder(firstUnsorted, a, first, unsorted - 1); } // end for } // end insertionSort "Insert" each item in array into its correct spot private static <T extends Comparable<? super T>> void insertInOrder(T element, T[] a, int begin, int end) { int index; for (index = end; (index >= begin) && (element.compareTo(a[index]) < 0); index--) a[index + 1] = a[index]; // make room } // end for a[index + 1] = element; } // end insertInOrder Find correct spot for current item

Lecture 17: Simple Sorts The code is a bit wordy – the authors present it in this way to be more readable Idea: Initial method has only array and length as params This calls an overloaded version with start and end index values as params – allows us to sort only part of the array if we want Each iteration in this method brings one more item from the unsorted portion of the array into the sorted portion It does this by calling another method to actually move the value into its correct spot Values are shifted from left to right, leaving a "hole" in the spot where the item should be

Run-time of InsertionSort?
Lecture 17: Simple Sorts Run-time of InsertionSort? Key instruction: comparisons of array items What is the WORST possible case scenario? Consider each iteration of the insertionSort loop when unsorted = 1, 1 comparison in insertInOrder method when unsorted = 2, 2 comparisons in insertInOrder method … when unsorted = N-1, N-1 comps in insertInOrder method Overall we get … + N-1 = (N-1)(N)/2 Considering Big O, we have O(N2) This is the worst case On average, the actual comparisons are a bit better, but it is still O(N2) Worst case scenario: Data is initially reverse-sorted

Can we use InsertionSort on a linked list?
Lecture 17: Simple Sorts Can we use InsertionSort on a linked list? What do you think? Yes – in fact it is probably more natural with a linked list At each iteration simply remove the front node from the list, and "insert it in order" into a second, new list In this case we are not creating ANY new nodes – just moving the ones we have around Do demo no board Run-time? Same run-time, but interestingly, the worst case situation is the opposite of that for the array version Discuss See Section 8.15 in text

Two other well-known simple sorts:
Lecture 17: Simple Sorts Two other well-known simple sorts: SelectionSort At iteration i of the outer loop, find the ith smallest item and swap it into location i i = 0 : find 0th smallest and swap into location 0 i = 1 : find 1th smallest and swap into location 1 … i = N-1 : find (N-1)th smallest and swap into loc N-1 Also a very simple implementation using nested for loops (or method calls, as shown in text) We saw this algorithm earlier in the term with Example2.java (go back and look at SortArray.java) See example on board

BubbleSort Lecture 17: Simple Sorts Item j is compared to item j+1
If data is sorted, item j should be less than item j+1 In this case we do nothing If item j is greater than item j+1, they are out of order In this case we swap them Continue from beginning again until sorted 1 2 3 4 5 6 50 30 40 70 10 80 20 Table above shows ONE iteration of the outer loop – here the largest item (80) is put into its correct location. The next iteration of the outer loop will put 70 into its correct location (index 5), and so on

Lecture 17: Simple Sorts Text also discusses recursive implementations of InsertionSort and SelectionSort As with Sequential Search and some other simple problems, this is more to show how it can be done rather than something that we would actually do Since these each have only a single recursive call and are not divide and conquer, there is no efficiency or implementation motivation to doing them recursively Read over these explanations and convince yourselves that the recursive versions do the same thing as the iterative versions

SelectionSort also has O(N2) run-time
Lecture 17: Simple Sorts SelectionSort also has O(N2) run-time Note that all of these simple sorting algorithms have similar run-times in the worst case InsertionSort – O(N2) SelectionSort – O(N2) BubbleSort – O(N2) For a small number of items, their simplicity makes them ok to use But for a large number of items, this is not a good run-time We'd like to come up with something better

Let's again consider InsertionSort
Lecture 18: Shellsort To improve on our simple sorts it helps to consider why they are not so good Let's again consider InsertionSort What about the algorithm makes its performance poor? Consider what occurs with each comparison Either nothing (if items are relatively in order) Or a data move of 1 location i.e. it only moves a small amount If the data is greatly out of order, it will take a lot of comparisons to get into order

This is the idea of Shellsort
Lecture 18: Shellsort If we can move the data farther with one com-parison, perhaps we can improve our run-time This is the idea of Shellsort Rather than comparing adjacent items, we compare items that are farther away from each other Specifically, we compare and "sort" items that are K locations apart for some K i.e. We Insertionsort subarrays of our original array that are K locations apart We gradually reduce K from a large value to a small one, ending with K = 1 Note that when K = 1 the algorithm is straight Insertionsort

Lecture 18: Shellsort 1 2 3 4 5 6 7 40 20 70 60 50 10 80 30 K = 4 40 10 70 30 50 20 80 60 K = 2 40 10 50 20 70 30 80 60 K = 1 The idea is that by the time K = 1, most of the data will not have very far left to move

Yet, when timed it actually outperforms Insertionsort
Lecture 18: Shellsort It seems like this algorithm will actually be worse than Insertionsort – why? It's last "iteration" is a full Insertionsort Previous iterations do Insertionsorts of subarrays Yet, when timed it actually outperforms Insertionsort Exact analysis is tricky, and depends on initial value for K Insertionsort actually has a very good run-time (O(N)) in the best case – Shellsort moves the data toward this best case A good implementation will have about O(N3/2) execution Compare to N2 for large N See text for more details

Lecture 18: Shellsort public static <T extends Comparable<? super T>> void shellSort(T[] a, int first, int last) { int n = last - first + 1; // number of array elements int space = n / 2; // initial gap is n/2 // Continue until the gap is zero while (space > 0) // Insertionsort all of the subarrays determined by // the current gap for (int begin = first; begin < first + space; begin++) incrementalInsertionSort(a, begin, last, space); } space = space / 2; // reduce gap } // end for } // end shellSort

Lecture 18: Shellsort private static <T extends Comparable<? super T>> void incrementalInsertionSort(T[] a, int first, int last, int space) { int unsorted, index; for (unsorted = first+space; unsorted<=last; unsorted=unsorted+space) T nextToInsert = a[unsorted]; index = unsorted – space; while ((index >= first) && mextToInsert.compareTo(a[index] < 0)) a[index + space] = a[index]; index = index – space; } // end while a[index + space] = nextToInsert; } // end for } // end incrementalInsertionSort

Lecture 18: Improved Sorts
Even Better Sorting Algorithms If we approach sorting in a different way, we can improve the run-time even more How about using Divide and Conquer? General Idea: Define sorting an array of N items in terms of sorting one or more smaller arrays (for example, of size N/2) As we said previously (for Binary Search), this works well when implemented using recursion So we will look at the next two sorting algorithms recursively

Lecture 18: Divide and Conquer Sorts
How can we apply D and C to sorting? Questions to consider: How do we "divide" the problem into subproblems? Do we break the array in half, or in some other fragment? Do we break it up by index value, or in some other way? How do we use the solutions of the subproblems to determine the overall solution? Once our recursive call(s) complete, what more needs to be done (if anything) to complete the sort? Let's examine these questions for MergeSort and QuickSort, two famous D and C sorting algorithms

Lecture 18: Idea of MergeSort
How do we "divide" the problem? Simply break the array in half based on index value Given the initial array We divide it into We then recursively divide each side, getting 1 2 3 4 5 6 7 40 80 60 20 30 10 70 50 1 2 3 4 5 6 7 40 80 60 20 30 10 70 50 1 2 3 4 5 6 7 40 80 60 20 30 10 70 50

We continue recursively until we reach the base case We know any array of size 1 is sorted already In the case below, we have 8 "arrays", each of size 1 Recall that physically, however, we still have only 1 array The subarrays are determined by index restrictions Once the base case is reached, we have to determine how to "put the pieces back together again” This is where we are actually doing the “work” 1 2 3 4 5 6 7 40 80 60 20 30 10 70 50

How do we use subproblem solutions to solve the overall problem? When the recursive calls complete, we will have two sorted subarrays, one on the left and one on the right Let's look at this from the first call's point of view How do we produce a single sorted array from these two sorted subarrays? 1 2 3 4 5 6 7 20 40 60 80 10 30 50 70

We "merge" them together, moving the next appropriate item into an overall sorted array Note that this is where we are really doing the "work" of the sort. We are comparing items and moving them based on those comparisons 1 2 3 4 5 6 7 20 40 60 80 10 30 50 70 1 2 3 4 5 6 7 10 20 30 40 50 60 70 80

Now we can look at pseudocode
Lecture 18: MergeSort Now we can look at pseudocode MergeSort(A) if (size of A > 1) Break A into left and right halves Recursively sort left half Recursively sort right half Merge sorted halves together Looking at the pseudocode, the algorithm seems pretty easy The only part that requires some thought is the merge Discuss code and do a trace in class

Lecture 19: MergeSort Runtime
Look at TextMergeQuick.java, trace merge How long does MergeSort take to run? Consider an original array of size N The analysis is tricky due to the recursive calls Let's think of the work "level by level" At each level of the recursion we need to consider and possibly move O(N) items Since the size is cut in half with each call, we have a total of O(log2N) levels Thus in total we have N x log2N work to do, so our runtime is O(Nlog2N) Note that when multiplying Big-O terms, we do NOT throw out the smaller terms Discuss this on the board Formal run-time analysis can be done for MergeSort using recurrence relations. We will do it informally here.

Lecture 19: MergeSort Runtime
Keep in mind that we are looking at MergeSort "level by level" simply to do the analysis The actual execution of MergeSort is a tree execution, similar to what we did for Hanoi Note that we recursively sort the left side of the array, going down all the way to the base case, and then merging back, before we even consider the right side See trace in next slide Yet we know Towers of Hanoi required 2N-1 moves while MergeSort only requires O(NlgN) comparisons Why this difference? Recall how the problem size decreases: Towers of Hanoi  N-1 MergeSort  N/2

Lecture 19: MergeSort Tree Execution Trace
START END : merge subarrays together

Lecture 19: MergeSort vs. Towers of Hanoi Run-Time
MergeSort Asymptotic Analysis Execution Trace has ~lgN levels Each call of size X spawns 2 calls of size X/2 At each call of size X we have to do X work (merge) This leads to a sum of N + 2(N/2) + 4(N/4) + … If we assume N = 2K for some K, this leads to 202K + 212K K-2 + … + 2K20 = (K+1)2K But we know K = lg2N so we get (lg2N+1)N => O(Nlg2N)

Lecture 19: MergeSort vs. Towers of Hanoi Run-time
Towers of Hanoi Asymptotic Analysis Execution Trace has exactly N levels Each call of size X spawns 2 calls of size X-1 At each call of size X we have to do 1 move This leads to a sum of … Or … + 2N-1 = 2N – 1 => O(2N) Even though the execution traces look similar, the run-times are NOT

Lecture 19: MergeSort Overhead
MergeSort's runtime of O(Nlog2N) is a definite improvement over our primitive sorts However, in order to "merge" we need an extra array for temporary storage We are NOT sorting in place here This adds memory requirements Although O(N) memory these days is not that big of a deal More importantly, copying to and from this extra memory slows down the algorithm in real terms The asymptotic runtime is very good, but when actually timed in practice we can do better Let's try another approach: QuickSort

Lecture 19: Idea of QuickSort
How do we "divide" the problem? QuickSort takes a different approach Instead of using index values to divide, break up the data based on how it compares to a special data value, called the pivot value. We compare all values to the pivot value, and place them into 3 groups: Since we are dividing by comparing values to another value, note that the division may NOT be exactly in half Data <= Pivot Pivot Data >= Pivot

Let's look at an example: Same original data as MergeSort example Now the "divide" has a different result Before we can divide, we need to choose the pivot value Can be any item – let's make it the last one, or A[last] We will later see a better way to do this In this case it is A[7] or the value 50 However, at the end of the "divide", the pivot may end up in a different index, since it should be "between" the two sides 1 2 3 4 5 6 7 40 80 60 20 30 10 70 50

Let's call this dividing partition Partition of our data using 50 as the pivot yields: We will see how partition is implemented shortly What does this achieve? Certainly the data is not yet sorted However, now we know that at least 1 item in the array is in its CORRECT, sorted location Which one? The rest of the data is now "more sorted" than it was, since it is at least on the correct "side" of the array 1 2 3 4 5 6 7 40 10 30 20 50 80 70 60 <= pivot pivot >= pivot The pivot item is in its correct location after partition

Naturally, the "divide" is not complete without recursive calls For QuickSort, we can now recursively sort the left "side" and the right "side" Recall that these sides may not be exactly ½ of the array We are now ready for pseudocode: QuickSort(A) if (size of A > 1) Choose a pivot value Partition A into left and right sides based on the pivot Recursively sort left side Recursively sort right side

How do we use subproblem solutions to solve the overall problem? We don't have to do anything! Note that we are comparing during partition Since the pivot is already in its correct spot, if we recursively sort the left side and we recursively sort the right side, the whole array is sorted So even though we need to consider 2) here, we don't need to do anything to accomplish it (unlike MergeSort) However, implementing 1) for QuickSort requires much work, also unlike MergeSort

So how is the partition done?
Lecture 19: QuickSort So how is the partition done? We'd like to do this in place if possible No extra array/vector needed Let's look at the code and trace the example on the board See Quick.java Note that this is still a simple version We will look at the text version after we discuss the run-time Recall that Merge from MergeSort requires an extra array/vector

Lecture 19: QuickSort Partition: basic idea Start with a counter on the left of the array and a counter on the right of the array As long as data at left counter is less than the pivot, do nothing (just increment counter) As long as the data at right counter is greater than the pivot, do nothing (just decr. counter) Idea here is that data is already on the correct side, so we don't have to move it When left counter and right counter "get stuck", it means there is data on the left that should be on the right, and vice versa So swap the values and continue

A[b] is greater than the pivot, but on left
Lecture 19: QuickSort 1 2 3 4 5 6 7 40 80 60 20 30 10 70 50 b c A[b] is greater than the pivot, but on left A[c] is less than the pivot but on right Swapping them puts things straight A[b] is again greater than the pivot A[c] is again less than the pivot Swap again to put things straight INITIALLY: left = 0 right = 7 pivot = 50 pivotIndex = 7 b = 0 (indexFromLeft) c = 6 (indexFromRight) 1 2 3 4 5 6 7 40 10 60 20 30 80 70 50 b c Since A[7] is the pivot, we start from the right at 6, or one position to the left of the pivot. We will handle the pivot index at the end of partition

Lecture 19: QuickSort 1 2 3 4 5 6 7 40 10 30 20 60 80 70 50 b c The values again are on the "wrong side", but this time note that b >= c This means we are done with the partition except for one last step – what is that? We must put the pivot into the right place Swap A[pivotIndex] and A[indexFromLeft] (A[b]) Set pivotIndex = indexFromLeft 1 2 3 4 5 6 7 40 10 30 20 50 80 70 60

Lecture 19: QuickSort Now we recursively sort the left side (blue) and recursively sort the right side (orange), and we are finished Note that the pivot from this first partition is never again touched – it is in its absolute correct spot The other items, however, could move considerably within their sides of the array 1 2 3 4 5 6 7 40 10 30 20 50 80 70 60

How long does QuickSort take to run?
Lecture 19: QuickSort How long does QuickSort take to run? The performance of QuickSort depends on the "quality" of the divide Depends on how other values relate to the pivot Let's look at 2 different scenarios: Pivot is always the middle value in a partition Show on board This execution trace is similar to that of MergeSort, and the overall Big-O runtime is also O(Nlog2N) However, since an extra array is NOT needed in QuickSort, the measured runtime will usually be faster than MergeSort

Pivot is always an extreme element in a partition
Lecture 20: QuickSort Pivot is always an extreme element in a partition Note that this is not the index of the pivot, but rather where the pivot ends up after the partition is complete Show on board Develop and discuss run-time Recall the idea of divide and conquer Recursive calls are a fraction of original size (ex: ½) However, in this case the recursive calls are only one smaller than the original size (N-1) Thus we are losing the power of divide and conquer in this case Run-time ends up being O(N2) Same as the simple sorts – see notes below Big-O of QuickSort in this case is O(N2), similar to that of the simple sorts: In first call partition must compare N-1 items to the pivot In second call partition must compare N-2 items to the pivot (note in this case each call only spawns 1, not 2 recursive calls) In third call partition must compare N-3 items to the pivot … Resulting sum is … + N-1 = (N-1)(N)/2  O(N2)

So which run-time will we actually get?
Lecture 20: QuickSort So which run-time will we actually get? It depends on how the data is originally distributed and how the pivot is chosen Our simple version of Quicksort picks A[last] as the pivot This makes the interesting worst case of the data being already sorted! The pivot is always the greatest element and there is no data in the “greater than the pivot” partition Reverse sorted data is also a worst case (now the pivot is always the smallest item) However, for “random” data it should perform well since it is not likely that poor pivots will consistently be chosen

Now reconsider already sorted data
Lecture 20: QuickSort We can make the worst case less likely to occur by choosing the pivot in a more intelligent way The text version uses Median of Three Median of Three Idea: Don't pick the pivot from any one index Rather consider 3 possibilities each time we partition A[first], A[mid], A[last] Order these items and put the smallest value back into A[first], the middle into A[mid] and the largest into A[last] So now we know that A[first] <= A[mid] <= A[last] Now use A[mid] as the pivot Now reconsider already sorted data Now it is a best case!

Lecture 20: QuickSort However, median of three does not guarantee that the worst case (N2) will not occur If only reduces the likelihood and makes the situation in which it would occur not obvious So we say: The EXPECTED run-time of QuickSort is O(Nlog2N) The WORST CASE run-time of QuickSort is O(N2) These are true for both simple pivot and median of three For code, see TextMergeQuick.java

Other variations / optimizations:
Lecture 20: QuickSort Other variations / optimizations: What if we choose the pivot index randomly? For each call, choose a random index between first and last (inclusive) and use that as the pivot Worst case? Could be just as bad as the simple pivot choice Average case? It is very unlikely that a random pivot will always be bad Do math on board Overall this should give good results However, we have overhead of generating random numbers Prob. that 1st pivot is extreme = 2/N Prob. that 2nd pivot is also extreme = 2/(N-1) Prob. that 3rd pivot is also extreme = 2/(N-2) … Total prob. (multiply for "and") = 2N/N! 2N may seem to be a very large number but compared to N! it is actually very small. Plug in some values to see.

What if we choose more than one pivot?
Lecture 20: Quicksort What if we choose more than one pivot? Dual pivot Quicksort: Use 2 pivots P1 and P2 and create three partitions: The items that are < P1 The items that are >= P1 and <= P2 The items that are > P2 This yields 3 subarrays that must be sorted recursively As long as pivots are chosen wisely, this actually has an incremental improvement over traditional QuickSort This has been incorporated into JDK in Java 7 (& 8) See:

When to stop recursion? Simple QuickSort stops when logical size is 1
Lecture 20: QuickSort When to stop recursion? Simple QuickSort stops when logical size is 1 However, benefit of divide in conquer decreases as problem size gets smaller At some point, the cost of the recursion outweighs the D and C savings So choose a size > 1 to stop recursing and switch to another (good) algorithm at that point What to choose? InsertionSort!!! Why? For very small arrays data does not have to move far and InsertionSort should work well See TextMergeQuick.java

Lecture 20: Quicksort Alternatively: Stop at base case > 1 but do NOT sort the items in the recursive call at all After all recursion is complete, InsertionSort entire array Even though it is poor overall, if the data is “mostly” sorted due to QuickSort, we will be close to the best case for InsertionSort and maybe we will get better overall results! It would be interesting to time all of these variations to see which has the best performance!

Lecture 20: QuickSort vs MergeSort
So which do we prefer, MergeSort or QuickSort? MergeSort has a more consistent runtime than QuickSort However, in the normal case, QuickSort out-performs MergeSort Due to the extra array and copying of data, MergeSort is "normally" slower than QuickSort This is why many predefined sorts in programming languages are actually QuickSort Ex: In JDK Arrays.sort() for primitive types uses QuickSort (Dual Pivot version)

Lecture 20: QuickSort vs. MergeSort
However, Quicksort is not a stable sort Given two equal items, X1 and X2 where X1 appears before X2 in the original data After MergeSort, X1 will still be before X2 This is not guaranteed in Quicksort So (for example): Arnold Arnoldson has a salary of Mort Mortenson has a salary of Assume with a lot of other people the data is sorted first alphabetically and then by salary So from the salary sort point of view Arnold and Mort are equal With a stable sort Arnold should appear before Mort but with an unstable sort they could be reversed

Lecture 20: Quicksort vs Mergesort
Thus, for complex (Object) types, it may be better to use Mergesort even if it is a bit slower Ex: Java Up through JDK 6 Java used MergeSort for objects and Quicksort for primitive types Stability does not matter for primitive types so use the faster sort Stability does matter for objects so use the stable (but slower) sort In JDK 7 (and JDK 8) they switched to TimSort which is MUCH more complicated but a bit faster and still stable It is derived somewhat from Mergesort but actually incorporates several different approaches See:

Lecture 20: Quicksort vs. Mergesort
Let’s test this to verify! Note: We must be careful to make sure MergeSort is in fact stable: Note the relevant code in merge(): for (; (beginHalf1 <= endHalf1) && (beginHalf2 <= endHalf2); index++) { if (a[beginHalf1].compareTo(a[beginHalf2]) <= 0) tempArray[index] = a[beginHalf1]; beginHalf1++; } else tempArray[index] = a[beginHalf2]; beginHalf2++; Consider comparison of items. What makes it stable? What would make it NOT stable? If we change the operator to "<" then an equal item would be taken from the right side rather than the left. This would switch equal items and cause the algorithm to not be stable.

QuickSort vs. MergeSort
We will use (stable) Mergesort and (unstable) Quicksort with a comparator so we can sort objects on different fields Much of the background for this is beyond the scope of this course However, we can see the difference in Mergesort and Quicksort when data are “equal” in a field See People.java, MergeQuickComparator.java and Stability.java

Lecture 21: Quicksort vs. Mergesort
What about sorting a linked list? Mergesort works but there is more overhead Ex: How to divide a linked list in half? Takes O(N) to do this but is constant with an array However, it does not add any extra asymptotic work The work required in each “call” for the Merge method is also O(N) Splitting just also takes O(N) for the linked list See LListWithSort.java Quicksort also could work but Would require doubly linked list Partition overhead would be more than is worthwhile This is neat stuff!

Recall what the ListInterface (and List) is
Lecture 21: Iterators Recall what the ListInterface (and List) is A set of methods that indicates the behavior of classes that implement it Nothing is specified about how the classes that implement List are themselves implemented The data could be stored in an array, as in the author's AList class The data could be stored in a linked list, as in the author's LList class The data could be stored in some other way

Lecture 21: Iterators Question: How can users of any List class access the data in a sequential way? We could copy the data into an array and return the array – then we can access the array This is what the toArray() method does Can we do it without having to make a new array? An iterator is a program component that allows us to iterate through a list in a sequential way, regardless of how the list is implemented The details of HOW we progress are left up to the implementer The user of the interface just knows it goes through the data

Iterators are good for two main reasons:
Lecture 21: Iterators Why do we need these? What good are they? We will see that the implementation can be a bit convoluted, leading to questions like "are these things really worth while?" Iterators are good for two main reasons: They allow multiple "iterations" to co-exist on the same underlying object They can tailor the implementation of the iteration to the underlying data structure, without requiring the client to know it

Multiple "co-existing" iterations
Lecture 21: Iterators Multiple "co-existing" iterations Consider the following situation: We have a set of data and we want to find the mode of that set What is the mode? How can we do this? Start at the first value See how many times it occurs – i.e. search through the rest of the list Proceed to the next value Do the same Continue all the way through, keeping track of the value with the highest count

Lecture 21: Iterators Show on board Note that we have two separate "iterations" through the list being accessed in the same code One is going through the list, identifying each item The other is counting the occurrences of that item Logically, they are separate, even though they are progressing through the same list For a List, we can also do this with nested for loops and the getEntry() method However, the implementation of getEntry() is very inefficient for a linked list As we discussed in Example8.java This leads us to the next point…

Tailor the implementation to the data structure
Lecture 21: Iterators Tailor the implementation to the data structure Consider again Example8.java When printing out either the AList or the LList, we use getEntry(i) to get the next item For the AList this is fine, since we have direct access to the locations However, for the LinkedList this is TERRIBLE getEntry(1) – 1 operation getEntry(2) – 2 operations getEntry(3) – 3 operations …

As we discussed, this gives us 1 + 2 + 3 + …
Lecture 21: Iterators As we discussed, this gives us … (our favorite sum!) Result is N(N+1)/2 => O(N2) for list of size N Why is it so poor for a linked list? Each getEntry() operation restarts at the beginning of the list What if we could "remember" where we stopped the last time and resume from there the next time? An iterator tailored to a linked list can do this for us, thereby saving a LOT of time Show on board

Consider the following methods:
Lecture 21: Iterators Consider the following methods: public boolean hasNext(); See if there are any elements remaining to iterate through public T next(); Retrieve and return the next element in the sequence, advancing the iterator by one position public void remove(); Remove the last item that was returned (via a call to next()) from the underlying data structure Consider these separate from any other functionality that a given class might have So we will make them an interface

Consider the Java Iterator interface:
Lecture 21: Iterators Consider the Java Iterator interface: public interface Iterator<T> { public boolean hasNext(); public T next(); public void remove(); } This is a simple iterator that can be used with most Collections But how is this interface implemented? Also, where is it implemented? We want it to be part of a List, but how can that be done, since List is itself an interface? This is a bit convoluted, so we need to consider this carefully Since Java 1.8 the Iterator also has the default method forEachRemaining(). See Java documentation for information on this method.

There are two ways we can implement this interface:
Lecture 21: Iterators There are two ways we can implement this interface: Internally: A list includes these methods amongst the other methods that it already has This solves problem 2) because we can tailor the implementation to the underlying class However, it does NOT solve problem 1) since we still only have one "state" available in the iteration Discuss Externally: A new object is created "on top" of the list that implements these methods Our “List” has only one set of instance variables, so it can keep the state for only 1 iteration

Lecture 21: Iterators We write our list classes so that each has the ability to generate an iterator object that allows sequential access to its elements, without violating data abstraction Thus the iterator object is separate (but related to) the underlying list that it iterates over Multiple iterator objects can be created for a given list, each with its own current "state" The external implementation will thus be preferable and is the technique that is used in standard Java, so we will look at this one in more detail

Idea: We only add a single extra method to our List interface:
Lecture 21: Iterators Note: This code depends heavily on object-oriented ideas and coding, so keep that in mind Idea: We only add a single extra method to our List interface: public Iterator<T> getIterator() This will return an iterator built on top of the current list, but with its own "state" so that multiple iterators can be used on one list Let's look at that method for the linked list implementation:

Lecture 21: Iterators public Iterator<T> getIterator() { return new IteratorForLinkedList(); } // end getListIterator So this method is easy – the work is in creating the new class IteratorForLinkedList This class will be built on the current list and will simply have the ability to go through all of the data in the list in an efficient way Since it is tailored to the linked list, we can make it a private (inner) class and it can directly access our linked list instance variables Let's look at the details in handout See Example12.java and LinkedListWithIterator.java

Let's now focus on the implementation
Lecture 21: Iterators Let's now focus on the implementation Recall that we said the iterator could be tailored to the underlying list The interface is the same, but the way it is done depends on whether the list is implemented with an array or a linked list The LL implementation uses a Node reference as the sole instance variable for the iterator It is initialized to firstNode when the iterator is created It progresses down the list with each call to next() Note that with a single Node reference, remove() is not possible Why? Discuss

So what would we need for the array implementation?
Lecture 21: Iterators So what if we wanted to allow remove()? We would need a second reference to keep track of the previous node in the iteration This is what is done in the Standard Java LinkedList iterator So what would we need for the array implementation? Discuss We need only an integer to store the index of the current value in the iteration It is incremented with each call to next() remove() can be implemented Must shift to fill in gap

Lecture 21: ListIterator
The Iterator interface can be used for any Java Collection This includes our List<T> interface, but also others: Ex: Set<T>, SortedSet<T> For a List, we can add more functionality to our iterator Basically we can traverse in both directions rather than one direction only Does this have any implications on our implemetations? Singly Linked List will not support a ListIterator!

public interface ListIterator<T> extends Iterator<T> { boolean hasNext(); T next(); boolean hasPrevious(); T previous(); int nextIndex(); int previousIndex(); void remove(); void set(T o); void add(T o); } Note that this iterator is bidirectional, and it allows objects to be added or removed

As we discussed previously for Iterator, the best way to implement a ListIterator is to Implement it "externally", meaning that the methods are not part of the class being iterated upon We build a ListIterator object on top of our list so we can have multiple iterations at once Make the class that implements the ListIterator an inner class so that it has access to the list details Allows us to tailor our ListIterator to the underlying data structure in the most efficient way However, we need a bit more logic to handle traversal in both directions, as well as both set() and remove()

Regarding the logic It is explained in great detail in the text – read it over See Chapter 15 Basic idea is that we need to be able to go in both directions in the list, so we need additional instance methods We will not focus on this implementation, but look it over in the text

Another interesting issue: The structure of iterators allow for multiple iterations on the same underlying list However, if we start modifying the underlying list, we can get into a lot of problems If one iterator modifies the list it will affect the other, and it could lead to an exception Because of this, the Standard Java iterators do not allow "concurrent modification" If one iterator modifies the list, other current iterators are invalidated, and will generate an exception if used

Lecture 22: Iterable Interface
With JDK 1.5, the Iterable interface was introduced This is simply: public interface Iterable<T> { Iterator<T> iterator(); } So any class with an iterator can also implement Iterable See MyArrayIterable.java and Example3b.java The benefit of this is that these classes can be used with the Java “foreach” loop – cool! In Java 8 Iterable has two additional default methods: forEach() and spliterator()

Lecture 22: Intro to Trees
Consider the primary data structures that we have examined so far: Array, ArrayList, LinkedList (also informally Stack and Queue – more on these later) All of these have been LINEAR data structures Data is organized such that items have a single predecessor and a single successor Except first (no predecessor) and last (no successor) We can draw a single "line" through all elements Note: We also covered the Bag, which is not necessarily linear, but both of our implementations were linear These data structures have worked well, but… Can we benefit from organizing the data differently? We will come back and talk about Stacks and Queues later

Tree structures In a linked list, each node had a reference to at most one previous and one next node What if we allowed nodes to have references to more than one next node? Root Node – has no parent node Interior Node – has a parent and at least one child node Doubly linked lists have a previous reference – we mentioned them briefly Leaf Node – has no children

A tree is a non-linear data structure, since we cannot draw a single line through all of the elements Some more definitions: For any node V, if P = parent(V), then V = aChild(P) For any node V, the descendants of V are all nodes that can be reached from V Parent(V) V siblings (all have same parent) s s s Descendants of V

For any node V, the subtree rooted at V is V and all of its descendants From V's point of view this is a tree in itself Now we can define a tree recursively: T is a tree if T is empty (no nodes) – base case, or T is a node with 0 children – base case or 1 or more children that are all trees – recursive case Do example on board

How do we represent an arbitrary tree? We can have a node with data and an adjacency list of children Draw on board Note that the number of children can be arbitrary List could be long if node has many children We can have a node with data, and two references, One to left child and one to right sibling Number of children can still be arbitrary List can still be long Shows the levels of the tree better You will likely see these representations in more detail in CS 1501

In many applications, we can limit the structure of our tree somewhat
Lecture 22: Binary Trees In many applications, we can limit the structure of our tree somewhat BINARY TREE A tree such that all nodes have 0, 1, or 2 children Recursive definition: T is a binary tree if T is empty (base case) or T is a node with the following structure where element is some data value where left and right are binary trees left element right

Lecture 22: Binary Tree Properties
Consider a binary tree with n nodes: Height of the tree is the maximum number of nodes from the root to any leaf Tree to right has a height of 6 We can also think of heights of subtrees of trees Subtree rooted at X has a height of 3 Height is an important property Many binary tree algorithms have run- times proportional to the tree height Let's establish some bounds on height x Some definitions start at 0

Maximum Height: Given a binary tree with n nodes, what is the maximum value it could have for its height How would the maximum height tree look? Discuss and see notes below Minimum Height: Given a binary tree with n nodes, what is the minimum value it could have for its height? Assume for simplicity that n = 2k-1 for some k How would this minimum height tree look? How can we justify its height value? Discuss Maximum height tree would be a linear tree – every node has one child (except the last, sole leaf). This tree would have a height of n given an n-node tree.

A minimum height tree will have the maximum branching at each node Given n = 2k-1, this tree will be a Full Tree All interior nodes have 2 children All leaves are on the same, last level of the tree Ex: Tree on bottom of this slide is a full tree of height 3, and it has 23-1 = 7 nodes So how can we relate n (7) to the height (3)? Note the number of nodes at each level of a full tree: Level 1: 1 node = 20 Level 2: 2 nodes = 21 Level 3: 4 nodes = 22 … Level i: 2i-1 nodes The total number of nodes is the sum of the nodes at each level

Recall that n = 2k-1 for some k Recall (from the last slide) that n = … + 2h-1 for some h Note that h is the height of the tree, so if we can solve for h we are done Thus, we get 2k-1 = n = … + 2h-1 for some h Using math, we know that … + 2h-1 = 2h-1 (geometric sum) Now we have 2k-1 = 2h-1 Adding 1 to both sides we get 2k = 2h Taking the log2 of both sides we get h = k

But we want the height in terms of the number of nodes, n: 2k-1 = n 2k = n+1 k = log2(n+1) So the minimum height of a binary tree with n nodes = h = k = log2(n+1) Note this is for a tree with 2k-1 nodes Binary trees can have any number of nodes – will this change the formula? Not significantly More generally we can say that the minimum height for a tree with n nodes is O(log2n) Now we also know that a Full Tree of height h has 2h-1 nodes

Note that most trees CANNOT be Full Trees, since all Full Trees have 2i-1 nodes (1, 3, 7, 15, etc) However, a tree with ANY number of nodes can be a Complete Binary Tree A complete tree is a full tree up to the second last level with the last level of leaves being filled in from left to right If the last level is completely filled in, the tree is Full A Complete Binary Tree of height h has between 2h-1 and 2h-1 nodes A nice property of a complete binary tree is that its data can be efficiently stored in an array or vector Do demo on board We will see why this is nice when we discuss PQs

Lecture 23: Height of a Binary Tree
So how do we find out the height for a given tree? We can define this recursively as well: Height(T) If T is empty, return 0 else Let LHeight = Height of left subtree Let RHeight = Height of right subtree Return (1 + Max(LHeight, RHeight)) Let's look at an example Tree we looked at previously in Slide 277

Lecture 23: Height of a Binary Tree
Trace on board with class

Lecture 23: Representing a Binary Tree
We'd like to be able to do operations on binary trees Implement the height that we just discussed Traverse the tree in various ways Find other properties Max or min value Number of nodes Before we can do these we need to find a good way to represent the tree in the computer

We'll do this in an object-oriented way, as we did with our lists It is a bit complicated, so we need to pay attention to all of the steps public interface TreeInterface<T> { public T getRootData(); public int getHeight(); public int getNumberOfNodes(); public boolean isEmpty(); public void clear(); } Note that this interface is for general trees Let's make it more specific for binary trees

public interface BinaryTreeInterface<T> extends TreeInterface<T>, TreeIteratorInterface<T> { public void setTree(T rootData); public void setTree(T rootData, BinaryTreeInterface<T> leftTree, BinaryTreeInterface<T> rightTree); } This simply allows for an "easy" assignment of binary trees We'll look at TreeIteratorInterface<T> later Now we have the basic functionality of a binary tree – but we need to get the basic structure We'll talk about the iterators later

Recall our linked list data structures: The "building blocks" for our lists were Node objects that we defined in a different class This class could be separate For re-use / flexibility This class could be an inner class For access convenience We will do something similar for our binary trees We will define a BinaryNode class to represent the inner structure of our tree This will be more complex than our Node class for LLs because there are more things needed to manipulate our binary tree nodes

class BinaryNode<T> { public T getData(); public void setData(T newData); public BinaryNode<T> getLeftChild(); public BinaryNode<T> getRightChild(); public void setLeftChild(BinaryNode<T> newLeftChild); public void setRightChild(BinaryNode<T> newRightChild); public boolean hasLeftChild(); public boolean hasRightChild(); public boolean isLeaf(); public int getNumberOfNodes(); public int getHeight(); public BinaryNode<T> copy(); } Gives the basic functionality of a node Note lack of data – we will look at this soon

Summary so far: TreeInterface TreeIteratorInterface Give the basic functionality of a tree BinaryTreeInterface Adds a couple of methods for binary trees Interfaces give us the ADTs Now we need some classes to implement these interfaces BinaryNode class Gives us the underlying structure

class BinaryNode<T> { private T data; private BinaryNode<T> leftChild; private BinaryNode<T> rightChild; // See .java file for methods // (also in Slide 294) } Self-referential, just as linked list nodes However, can now branch in two directions Now we can easily define a binary tree We will also have some additional methods to manipulate / access our tree

public class BinaryTree<T> implements BinaryTreeInterface<T> { private BinaryNode<T> root; // See .java file for methods } Idea: A BinaryTree has one instance variable – a reference to a BinaryNode A BinaryNode has 3 instance variables An reference to T to store data for that node Left and right references to subtree nodes Creation by Composition To manipulate a BinaryTree we must manipulate its underlying nodes

We will come back to the BinaryTree<T> class later on For now we will look at the BinaryNode<T> class and see how some of the operations are done Finding the height, traversals, etc. We will then see how these operations will be used for our BinaryTree<T> class So consider the BinaryNode<T> class…

Lecture 23: Implementing Some Operations
Ok, let's first look at code that determines the height for a BinaryNode: public int getHeight() { return getHeight(this); } // call rec. version private int getHeight(BinaryNode<T> node) { int height = 0; if (node != null) height = 1 + Math.max(getHeight(node.getLeftChild()), getHeight(node.getRightChild())); return height; } Note that actual code is not really different from the pseudocode we looked at in Slide 288 and that we already traced

How about copying a tree? Copying an array or linked list is fairly simple, due to their linear natures However, it is not immediately obvious how to copy a binary tree such that the nodes are structurally the same as the original Luckily, recursion again comes to the rescue! If we view copying the tree as a recursive process, it becomes simple! To copy tree T, we simply Make a new node for the root and copy its data Recursively copy the left subtree into the left child Recursively copy the right subtree into the right child

Let's now look at code for copy(): public BinaryNode<T> copy() { BinaryNode<T> newRoot = new BinaryNode<T>(data); if (leftChild != null) newRoot.setLeftChild(leftChild.copy()); if (rightChild != null) newRoot.setRightChild(rightChild.copy()); return newRoot; } // end copy Note the similarities (and differences) to the code for getHeight() Both are essentially traversing the entire tree, processing the nodes as they go

Lecture 23: Trace of copy() method
BinaryNode<Integer> T2 = T1.copy() public BinaryNode<T> copy() { BinaryNode<T> newRoot = new BinaryNode<T>(data); if (leftChild != null) newRoot.setLeftChild(leftChild.copy()); if (rightChild != null) newRoot.setRightChild(rightChild.copy()); return newRoot; } // end copy T2 this newRoot newRoot.leftChild newRoot.rightChild T1 10 this newRoot newRoot.leftChild newRoot.rightChild 10 20 30 20 30 this newRoot 40 50 40 50 Note: View this in a ppt slideshow to see the animation

Lecture 24: Binary Tree Traversals
So what about traversing itself? Again, unlike linear structures (array, linked list) it is not obvious However, if we think recursively, we can still do it in a fairly easy way: Consider a tree node T I can traverse the subtree rooted at T if I Traverse T's left subtree recursively Visit T itself (i.e. access its data in some way) Traverse T's right subtree recursively left Data right Do an example on the board

There are 3 common traversals used for binary trees They are all similar – the only difference is where the current node is visited relative to the recursive calls PreOrder(T) if (T is not empty) Visit T.data PreOrder(T.leftChild) PreOrder(T.rightChild) InOrder(T) InOrder(T.leftChild) InOrder(T.rightChild)

PostOrder(T) if (T is not empty) PostOrder(T.leftChild) PostOrder(T.rightChild) Visit T.data Let's look at an example We'll traverse a tree using all 3 to see how it proceeds and what output it generates 50 Do traces on board 30 80 10 40 90 5 20 45 85 95

Note that in the example shown, the InOrder traversal produces the data IN ORDER This is NOT ALWAYS the case – it is only true when the data is organized in a specific way If the tree is a Binary Search Tree – we will see this later The actual code for these traversals is not any more complicated than the pseudocode See BinaryNode.java and Example14.java It uses one tree that is NOT a BST and one that is Note how the work is done through the recursive calls The run-time stack "keeps track" of where we are Runtime of these traversals? Discuss and see note below The runtime is O(N) since a recursive call is done for each of the N nodes and the instructions executed per node (other than the recursive calls) are constant (just a simple visit). Note that more than N calls of the traversal methods are actually done, since we have a call for each base case, which is an empty node. However, with N nodes in a binary tree, we will always have N+1 "empty" nodes (i.e. null references) so the Big-O value will still be O(N).

Note again how the traversals, getHeight() and copy() are all similar In fact all of these methods are traversing the tree They differ in the order (pre, in, post) and what is done at each node as it is visited For example: getHeight() can be thought of as a postorder traversal, since we have to get the height of both subtrees before we know the height of the root copy() is actually a combo of all 3 orderings The root node is created preorder The left child is assigned inorder The right child is assigned postorder

Can these traversals be done iteratively? Yes but now we need to "keep track" of where we are ourselves We do this by using our own stack of references The idea is that the "top" BinaryNode reference on our stack is the one we are currently accessing This works but it is MUCH MORE COMPLICATED than the recursive version The author uses the iterative versions these traversals to implement iterators of binary trees We will see how much harder these are to do iteratively However, we can't use the recursive version for an iterator, since it needs to proceed incrementally

Lecture 24: Binary Search Trees
Binary Trees are nice, but how can we use them effectively as data structures? One way is to organize the data in the tree in a special way, to create a binary search tree (BST) A BST is a binary tree such that, for each node in the tree All data in the left subtree of that node is less than the data in that node All data in the right subtree of that node is greater than the data in that node Note that this definition does not allow for duplicates. If we want to allow duplicates we should add "or equal to" to one of the above lines (but not both)

Naturally, we can also define BSTs recursively: A binary tree, T, is a BST if either T is empty (base case) or T is a node with the following structure where all values in the tree rooted at left are less than data where all values in the tree rooted at right are greater than data where left and right are BSTs left data right

50 BST 30 80 10 40 90 5 20 45 85 95 50 NOT A BST Note that the shape of the tree is unrelated to the BST property. BSTs can be full, linear or anywhere in between. BST 30 50 10 30 80 10 40 20 90 5

Lecture 24: BST Interface
Let's back up a bit now We haven't defined the BST ADT yet (i.e. the methods that make up a BST): Actually, the text defines a more general SearchTreeInterface, which our BST will implement: public boolean contains(T entry) Is an entry in the tree or not? public T getEntry(T entry) Find and return and entry that "equals" the param entry If the key matches return the object; otherwise return null

Lecture 24: BST Interface
public T add(T newEntry) Add a new entry into the tree New object is put into its appropriate location, keeping the search property of the tree intact If an object matching newEntry is already present in the tree, replace it and return the old object What if we don't want to replace it? Implications? public T remove(T entry) Remove entry from the tree and return it if it exists; otherwise return null public Iterator<T> getInorderIterator() Return an iterator that will allow us to go through the items sequentially from smallest to largest Go back and look at Iterator<T> interface

Before we discuss the implementation details
Lecture 24: BST Search Before we discuss the implementation details Let's get the feel for the structure by seeing how we would do the getEntry(T entry) method Consider a recursive approach (naturally): What is our base case (or cases)? If tree is empty – not found else if key matches node value -- found What are our recursive cases? If key < node value, search left subtree else if key > node value, search right subtree How do we use our recursive results to determine our overall results? Simply pass result from recursive call on Trace an example

Lecture 24: BST Search vs. Sorted Array Search
Notice the similarity between this algorithm and the binary search of a sorted array This is NOT coincidental! In fact, if we have a full binary tree, and we have the same data in an array, both data structures would search for an item following the exact same steps Let's look for item 45 in both data structures: 1 2 3 4 5 6 10 30 40 50 70 80 90 2 3 1 50 1 30 80 2 10 40 70 90 3

Lecture 24: BST Search vs. Sorted Array Search
In the case of the array, 45 is "not found" between 40 and 50, since there are no actual items between 40 and 50 In the case of the BST, 45 is "not found" in the right child of 40, since the right child does not exist Both are base cases of a recursive algorithm Same runtimes since the height of a full tree is O(log2n) Immediately, we see an advantage of the BST over the LinkedList Although access to nodes requires references to be followed, the tree structure improves our search time from O(n) to O(log2n) Ok, now is a BST also an improvement over the array?

Lecture 24: BST Implementation
To answer that question, we need to look at some more operations Let's first look more at the BST structure BST Implementation We will use the BinaryTree as the basis We can implement it either recursively or iteratively We'll look at both versions public class BinarySearchTree<T extends Comparable<? super T>> extends BinaryTree<T> implements SearchTreeInterface<T>

We will concentrate on four things: getEntry() method contains() can be easily derived from getEntry() add() method remove() method getInorderIterator() method These provide the basic functionality of a Binary Search Tree: Finding an object within the tree Adding a new object to the tree Removing an object from the tree Traversing the tree to view all objects

getEntry() We already discussed the idea of this method in a recursive way Now let's look at the actual code and trace it See recursive BinarySearchTree.java See iterative BinarySearchTree.java Note how iterations of the loop correspond to recursive calls See how contains() is easily derived

add() This one is more complicated Special case if tree is empty, since we need to create a root node Otherwise, we call addEntry(), which proceeds much like getEntry() However, we have more to consider. Consider possibilities at current node (call it temp): New data is equal to temp.data Store old value, assign new value and return old node New data is less than temp.data If temp has a left child, go to it else add a new node with the new data as the left child of temp

New data is greater than temp.data If temp has a right child, go to it else add a new node with the new data as the right child of temp Of course, the actual code is trickier than the pseudocode above Let's trace the recursive version to see how it works See recursive version of BinarySearchTree.java One interesting difference from getEntry()/findEntry() The base case for addEntry() must be at an actual node We cannot go all the way to a null reference, since we must link the new node to an existing node If we go to null we have nothing to link the new node to Thus we stop one call sooner for the base case for addEntry()

Lecture 25: BST Recursive addEntry() Method
Adding 25 to the BST Note: Run-Time Stack goes downward in this case rootNode 25<50, go left root rootNode 25<30, go left 50 30 80 rootNode 25>10, go right 10 40 90 rootNode 25>20, right null 5 20 45 85 95 Note: Author's code is a bit inconsistent with regard to local variables. When going left, a new variable, subTreeRoot is used to temporarily store the result of the recursive call. However, when going right, the rightChild variable is simply reassigned. 25 To see this correctly you must run it in a Powerpoint slideshow

Lecture 25: BST add() Method
This is elegant but it still it (obviously) requires many calls of the method As we know, this adds overhead to the algorithm If we do the process iteratively, this overhead largely goes away See iterative version Trace As with findEntry(), since the recursive calls are "either" "or" but not both, the iteration is very simple and actually preferred over the recursion We are not traversing the entire tree but rather just following a single path down from the root

Lecture 25: BST remove() Method
Idea is simple: 1) Find the node and, 2) Delete it However, it is much trickier than add – why? Unlike add(), which is always at a leaf, the remove() operation could remove an arbitrary node Depending upon where that node is, this could be a problem Let's look at 3 cases, and discuss the differences between them node has 2 children node has 1 child node is a leaf Trace on board

Node is a leaf This one is easy – simply set its parent's appropriate child reference to null (so we need a ref. to parent) Garbage collector takes care of the rest Node has one child Still not so bad…in fact this looks a lot like what? Deleting a node from a linked list Set parent's child reference to node's child reference Node has two children This one is tricky! Why -- only one reference coming in but two going out

So to actually delete the node would require significant reorganization of the tree But do we really even need to delete the NODE? No, we need to delete the DATA Perhaps we can accomplish this while leaving the node itself where it is How? Recall that what is important about a BST is the BST Property (i.e. the ordering) The shape is irrelevant (except for efficiency concerns, which we will discuss next) So perhaps we can move data from another node into the node whose value we want to delete Perhaps the other node will be easier to delete We saw a simplified form of this same idea very early in the term when we deleted from a LinkedBag

How do we choose this node? Consider an inorder traversal of the tree We could substitute the value directly before (inorder predecessor) or the value directly after (inorder successor) How to find this node? Consider inorder predecessor – it is the largest value that is less than the current value So we go to the left one node, then right as far as we can Show on board What if this node also has two children? Will not ever – since we know by how we found it that it has no right child

Let's look at the code to see how this is done We'll look at the iterative version Recursive version works, but due to the same issues we discussed for add(), we will prefer the iterative version Note that the code looks fairly tricky, but in reality we are just going down the tree one time, then changing some references A lot of the complexity of the code is due to the author's object-oriented focus

Lecture 25: Deleting a Node with 2 Children from a BST
50 30 25 80 10 40 90 5 20 45 85 95 30 is found It has two children Find Inorder Predecessor Go left Go right until null Overwrite current node with inorder predecessor Delete inorder predecessor 25 To see this correctly you must run it in a Powerpoint slideshow

Lecture 25: BST getInoderIterator() Method
getInorderIterator() As we discussed previously, this will be a step-by-step inorder traversal of the tree Recall idea of iterator from lists It is done iteratively so that we can pause indefinitely after each item is returned Still the logic is much less clear than for the recursive traversals This method is implemented in the BinaryTree class, so we don't have to add anything for BinarySearchTree See BinaryTree.java

Lecture 25: BST getInorderIterator() Method
What data and methods do we need? Method simply returns an instance of private InorderIterator object Recall the methods we need for an iterator() hasNext() – is there an item left in the iteration? next() – return the next item in the iteration We also need some instance variables To mimic the behavior of the run-time stack, we will use our own Stack object Plus we need a BinaryNode to store the current node How will it work? Think about behavior of inorder traversal We need to duplicate this iteratively

Initially (in the constructor), set the currentNode to the root For each call of next() Go left from root as far as we can, pushing all nodes onto the stack Top of the stack will be the next value in the iteration (nextNode) Then set the currentNode to the right child of nextNode After nextNode we should traverse the its right subtree That is what currentNode now represents It could be null – in this case the previous node had no right subtree, and we backtrack Let's trace this execution

nodeStack root 50 30 80 currentNode 10 40 90 nextNode 5 20 45 85 95 To see this correctly you must run it in a Powerpoint slideshow Trace is only partially shown (up to 40)

Lecture 26: BST Run-times
So how long will getEntry() (and contains()), add() and remove() take to run? It is clear that they are all proportional in run-time to the height of the tree So if the BST is balanced getEntry(), add() and remove() will all be O(log2N) If the BST is very unbalanced getEntry(), add() and remove() will all be O(N) Given normal use, the tree tends to stay balanced However, it could be unbalanced if the data is inserted in a particular way Ex: If we do add()s of sorted data from a file

Lecture 26: BST Run-times
Thus, in the AVERAGE CASE, BST give us O(log2N) for Find, Insert and Delete In the WORST CASE, BST gives us O(N) for Find, Insert and Delete So how does a BST compare to a Sorted array or ArrayList? Recall that a sorted array gives us (average) O(log2N) to find an item using binary search O(N) to add or remove an item (due to shifting) Thus, in the average case, BST is better for Insert and Delete and about the same for Find

Lecture 26: Balanced BSTs
"On average", a BST will remain balanced But it is possible for it to become unbalanced, yielding worst case run-times Can we guarantee that the tree remains balanced? Yes, for example the AVL Tree (Chapter 27) When Inserts or Deletes are done, nodes may be "rotated" to ensure that the tree remains balanced However, these rotations add overhead to the operations If we time the operations, on average it is actually slower than the regular BST

Lecture 26: Stacks One of the simplest and most commonly used data structures is the Stack Stack Data is added and removed from one end only (typically called the top) Logically the top item is the only one that can even be seen Think of a plate warmer in a buffet Fundamental Operations Push an item onto the top of the stack Pop an item from the top of the stack Peek at the top item without disturbing it See StackInterface.java

This access, although simple, is useful for a variety of problems
Lecture 26: Stacks A Stack organizes data by Last In First Out, or LIFO (or FILO – First In Last Out) This access, although simple, is useful for a variety of problems Let's look at a few applications before we discuss the implementation (i.e. we will don our “client” hats) Run-time Stack for method calls (esp. recursive calls) We have seen this with recursion When a method is called, its activation record is pushed onto the run-time stack When it is finished, its activation record is popped from the run-time stack

Testing for matching parenthesis
Lecture 26: Stacks Testing for matching parenthesis (()())() – match ((((())))) – match ((()) – don't match (not enough right parens) ())( – don't match (parens out of order, or too many right parens) ([)] – don't match (wrong paren type) How can we code this using a Stack? Let's solve this problem together Ok, what do we need: A character variable to store the current character A Stack (we need to figure out how it's used) A way to input the data

Discuss different cases and develop idea
Lecture 26: Stacks Discuss different cases and develop idea When do we push, when do we pop and how do we test? Let's consider the cases one at a time and see what we need to do to determine them Do on board Look at code: Driver.java & BalanceChecker.java From the Authors

Stacks can also be used to evaluate post-fix expressions
Lecture 26: Stacks Stacks can also be used to evaluate post-fix expressions Operators follow operands Useful since no parentheses are needed Ex: – 5 4 * / = ?? General algorithm? Idea is that each operator seen is used on the two most recently seen (or generated) operands So for example, the "–" is used on 10 and 6 So what do we do with operands before seeing an operator, or after we evaluate an intermediate result? Discuss and trace example on board Push operands onto the stack When an operator is encountered, pop the last two operands. perform the operation, and push the result back onto the stack

Lecture 26: Stacks We can also use a stack to convert from infix notation to postfix notation Ex: (a + b) * (c – d * e)  a b + c d e * – * This process is somewhat more complicated, since we need to be able to handle operands, operators (of different precedence) and possibly parentheses We will also need a StringBuilder (or StringBuffer) to store the result This process is discussed in detail in Section of the text Read over it carefully – it is explained quite well in the book

Lecture 26: Stacks Stack Implementation? A Stack can easily be implemented using either an array or a linked list Array: Push? Pop? See ArrayStack.java Linked List: See LinkedStack.java Array: Push item onto end of array, and increment index Array: Pop item from end of array, and decrement index Linked List: Push item onto front of list Linked List: Pop item from front of list

Lecture 26: Stacks In Java Collections Framework: class Stack extends class Vector, defining the Stack operations appropriately Look at API Note style problem: All Vector operations are still available, allowing user to violate Stack restrictions We will see later that the Queue ADT in Java is (more appropriately) an interface Difference is due to history and backward compatibility Discuss

Queue Data is added to the end and removed from the front
Lecture 26: Queues Queue Data is added to the end and removed from the front Logically the items other than the front item cannot be accessed Think of a bowling ball return lane Balls are put in at the end and removed from the front, and you can only see / remove the front ball Fundamental Operations enqueue an item to the end of the queue dequeue an item from the front of the queue front – look at the top item without disturbing it

Like a Stack, a Queue is a simple but powerful data structure
Lecture 26: Queues A Queue organizes data by First In First Out, or FIFO (or LILO – Last In Last Out) Like a Stack, a Queue is a simple but powerful data structure Used extensively for simulations Many real life situations are organized in FIFO, and Queues can be used to simulate these Allows problems to be developed and analyzed on the computer, saving time and money

Lecture 26: Queues Ex: A bank wants to determine how best to set up its lines to the tellers: Option 1: Have a separate line for each teller Option 2: Have a single line, with the customer at the front going to the next available teller How can we determine which will have better results? We can try each one for a while and measure Obviously this will take time and may create some upset customers We can simulate each one using reasonable data and compare the results Other (often more complex) problems can also be solved through simulation

Lecture 27: Queues Queue Implementation? We need a structure that has access to both the front and the rear We'd like both enqueue and dequeue to be O(1) operations We have two basic approaches: Use a linked-list based implementation Use an array based implementation We discussed these somewhat already Review slides on your own

Queue using a Linked List
Queues Queue using a Linked List This implementation is fairly straightforward as long as we have a doubly linked list or access to the front and rear of the list enqueue simply adds a new object to the end of the list dequeue simply removes an object from the front of the list Other operations are also simple We can build our Queue from a LinkedList object, making the implementation even simpler This is more or less done in the JDK See Queue and LinkedList in Java API

Queues Note that Queue is an interface The LinkedList class implements Queue (among other things) Note that in one way this is a good use of interfaces as ADTs Even though LinkedList can do a lot more than just the Queue operations, if we use a Queue reference to the object, we restrict it to the Queue operations Compare this to Stack, which was implemented as a class The text author also uses an interface, but implements the Queue from stratch See LinkedQueue.java from text Linked list with front and rear references is used

Are there other linked options?
Queues Are there other linked options? Recall that when we looked at linked lists, we considered a circular linked list The extra link gives us all the functionality we need for a Queue enqueue? newNode = new Node(newEntry, lastNode.next); lastNode.next = newNode; lastNode = newNode; dequeue? frontNode = lastNode.next; lastNode.next = frontNode.next; return frontNode; lastNode

The text takes this notion one step further:
Queues The text takes this notion one step further: Logic of enqueue and dequeue is the same However, when we dequeue, rather than removing the node (and allowing it to be garbage collected), we instead just "deallocate it" ourselves This way we save some overhead of creating new nodes all the time We keep two references: queueNode and freeNode queueNode is the front of the queue This will be the next node dequeued freeNode is the rear of the queue This will be the next node enqueued – if none left we will then create a new node Show on board and see TwoPartCircularLinkedQueue.java

Queues Queue using an array Arrays that we have seen so far can easily add at the end, so enqueue is not a problem Can clearly be done in O(1) time We may have to resize, but we know how to do that too However, removing from the front is trickier In ArrayList, removing from the front causes the remaining objects to be shifted forward This gives a run-time of O(N), not O(1) as we want So we will not use an ArrayList Instead we will work directly with an array to implement our Queue

How can we make dequeue an O(1) operation?
Queues How can we make dequeue an O(1) operation? What if the front of the Queue could "move" – not necessarily be at index 0? We would then keep a head index to tell us where the front is (and a tail index to tell where the end is) Ok…so now we can enqueue at the rear by incrementing the tail index and putting the new object in that location and we can dequeue in the front by simply returning the head value and incrementing the head index H T 30 80 60 40 70

What can we do to fix this problem?
Queues This implementation will definitely work, but it has an important drawback: Both enqueue and dequeue increment index values Once we increment front past a location, we never use that location again Thus, as the queue is used the data migrates toward the end of the array Clearly this is wasteful in terms of memory What can we do to fix this problem? We need a way to reclaim the locations at the front of the array without spending too much time So shifting is not a good idea Any ideas?

Queues How about proceeding down the array as we did before, but when we get to the end, we wrap around back to the beginning We call this a circular queue, since we use the array locations in a circular way Circular queue before enqueue of 80 Circular queue after enqueue of 80 H T 60 40 70 50 90 T H 80 60 40 70 50 90

Actually it is quite simple
Queues How can this be done? Actually it is quite simple When we increment the front and rear index values we do so mod the array length, or backIndex = (backIndex + 1) % queue.length; queue[backIndex] = newEntry; As long as backIndex+1 is less than queue.length, the result is a normal increment However, once backIndex+1 == queue.length, taking the mod will result in 0, returning us to the beginning of the array One remaining question: how do we know if the queue is empty or full?

Let's look at some more code
Queues Both indexes move throughout the array Show example on board front == (back+1) % queue.length when array is full or empty One easy solution is to keep track of the size with an extra instance variable Text doesn't want to do that (even though the size of a queue is often needed) Rather, they keep one location in the array empty, even if the queue is full Array is full when front == (back + 2) % queue.length Empty when front == (back + 1) % queue.length Let's look at some more code See ArrayQueue.java

Lecture 27: Priority Queues
Queues organize data FIFO Sometimes we want to remove data by other rules "Those traveling with small children may board" Tip the maitre d' to get a table Your Java program is running out of memory so the garbage collector needs to run This is the idea of a Priority Queue Data is removed by priority order, rather than FIFO.

Methods: Similar in nature to Queue add() an item to the PQ Similar to enqueue remove() and return the highest priority item Similar to dequeue peek() at the highest priority item Similar to getFront The difference is the order of the removals See PriorityQueueInterface.java

Implementation? Consider unsorted array: add()? peek()? remove()? Run-times? Consider sorted array: [see notes on bottom of slide] unsorted array: add at the end O(1) peek: search for smallest O(n) remove: search for smallest then delete it O(n) sorted array: add in correct spot O(lgn) to find spot, O(n) to shift peek: front or rear item: which is better? remove: simply remove that item [from end]

How about a linked-list? Unsorted will be similar to unsorted array Sorted does not buy us anything Why? For any of the above implementations, consider a sequence of N adds followed by N removes Let's figure out the total run-time and the amortized time per operation Do on board [Also see Notes on the bottom of this slide] Can we do better? Yes, with a HEAP In all cases with a simple array or linked list, one of the operations (either add or remove) is linear. Thus, if we consider N adds followed by N removes, the total run-time will be N^2 by the following logic [we consider the case of the unsorted array – other cases are similar] Each add() will be O(1) for a total of O(N) First remove will require N comparisons to find the highest priority item Second remove will require N-1 comparisons Third remove will require N-2 comparisons … Adding we get the sum: … + N which we know evaluates to N(N+1)/2 which is O(N^2) In the amortized case we have O(N^2)/N = O(N) per operation, which is more time than we want to spend

Lecture 27: Heaps Idea of a heap: Partial ordering of data in a logical complete binary tree For each node, T, in the tree: T.data has a higher priority than T.lchild.data T.data has a higher priority than T.rchild.data Note that NOTHING IS SAID about how T.lchild.data and T.rchild.data compare to each other We do not care – could be either way This is why it is a partial ordering Compare to BST, which is a complete ordering In that case, we define a specific relationship between siblings

Look at simple example on the board
Lecture 27: Heaps Higher priority here can mean either greater than or less than in terms of value Min Heap: Highest priority value is the smallest Ex: Seedings in an event, rankings, etc. Max Heap: Highest priority value is the largest Ex: Salary, batting average, goals per game, etc. The logic is the same for both Text uses Max Heap Look at HeapPriorityQueue.java and MaxHeapInterface.java We could very easily switch this to a Min Heap if needed Look at simple example on the board

Ok, how do we do our PQ / MaxHeap operations:
Lecture 27: Heaps Ok, how do we do our PQ / MaxHeap operations: peek() / getMax() is easy – ROOT of tree How about add and remove? add() / add() is not as simple remove() / removeMax() is even trickier For both we are altering the tree, so we must ensure that the HEAP PROPERTY is reestablished We need to carefully consider where / how to add and remove to keep the tree valid but also not cost too much work

Idea of add(): Idea of removeMax():
Lecture 27: Heaps Idea of add(): Add new node at next available leaf Push the node "up" the tree until it reaches its appropriate spot We'll call this upHeap See example on board Idea of removeMax(): We must be careful since root may have two children Similar problem exists when deleting from BST To delete that node will require a major reworking of the tree Instead of deleting root node, we overwrite its value with that of the last leaf

Lecture 27: Heaps Then we delete the last leaf -- easy to delete a leaf And we guarantee that the tree is still complete But now root value may not be the max Push the node "down" the tree until it reaches its appropriate spot We'll call this downHeap See example on board Note in both operations the node we add / remove is always the last leaf To maintain a complete tree However, the data is rearranged to also preserve the heap property

Run-time? Complete Binary Tree has height  lgN
Lecture 27: Heaps Run-time? Complete Binary Tree has height  lgN upHeap or downHeap at most traverse height of the tree Thus add() and removeMax are always O(lgN) worst case For N add and N removeMax operations: N x lgN = O(NlgN) Amortized the operations are (clearly) O(lgN) each This is definitely superior to either the array or linked list implementation

Lecture 27: Implementing a Heap
How to Implement a Heap? We could use a linked binary tree, similar to that used for BST Will work, but we have overhead associated with dynamic memory allocation and access To go up and down we need child and parent references Must keep track of "last leaf in tree" reference But note that we are maintaining a complete binary tree for our heap It turns out that we can easily represent a complete binary tree using an array

Lecture 27: Implementing a Heap
Idea: Number nodes row-wise starting at 1 Use these numbers as index values in the array Now, for node at index i See example on board Now we have the benefit of a tree structure with the speed of an array implementation See MaxHeap.java Parent(i) = i/2 LChild(i) = 2i RChild(i) = 2i+1

Lecture 27: Array vs. Linked List Implementations
So far we have discussed both array and linked list based data structures: For Bag we have ArrayBag and LinkedBag For List interface we have ArrayList (and Vector) and LinkedList For Stack we have subclass of Vector (as we discussed) or of LinkedList For Queue we have linked list version in text (LinkedQueue.java) and also the circular array-based version (ArrayQueue.java) So which do we prefer? It depends!

Consider Stack and Queue As long as resizing is done in an intelligent way, the array versions of these may be a bit faster than the linked list versions Stack: push(), pop() are O(1) amortized time for both implementations, but they are a constant factor faster in normal use with the array version Queue: enqueue(), dequeue() are O(1) amortized time for both implementations, but they are a constant factor faster in normal use with the array version But notice that the ArrayList does not automatically "downward" size when items are deleted, so the ArrayList-based Stack will not either It could waste memory if it previously had many items and now has few

In general, you need to decide for a given application which implementation is more appropriate In real life, however (especially now) Most of these data structures are predefined in a library Java Collections Framework Stack is array-based, Queue is LL-based C++ Standard Template Library It's still good to understand how they are implemented, but more often than not we just use the standard version, due to convenience

Lecture 28 Exam 2 Focus on material from Lecture 14-Lecture 27

Bonus Material: Mutable and Immutable Objects
Many classes that we build contain mutator methods Methods that allow us to change the content of an object Objects that can be changed via mutators are said to be mutable Ex: StringBuilder append() method adds characters to the current StringBuilder Ex: Rectangle2D.Double setFrame() method changes size and location

Ex: ArrayList add(), remove() for example Some classes do not contain mutator methods Objects from these classes are said to be immutable Ex: String Cannot alter the string once the object is created Ex: wrapper objects (Integer, Float, etc) Allow accessors but no mutators

Implications of Mutable vs. Immutable Objects Complications of being immutable Actions that could be simple as a mutation require more work if a new object must be created Ex: Concatenating Strings String S1 = "Hello "; S1 = S1 + "there"; We must create and assign a new object rather than just append the string to the existing object If done repeatedly this can cause a lot of overhead Show on board

Complications of being mutable Consider collections of objects When we add an object to a collection, it doesn't mean we give up outside access to the object If we subsequently alter the object "external" to the collection, we could destroy a property of the collection Ex: Consider a BST or a MaxHeap In either of these cases the data must meet a certain requirement based on its value Altering an object within the BST or MaxHeap could cause the collection to no longer satisfy the BST property or the Heap property

Bonus Material: Cloning
What to do? We can make objects immutable We can put clones of our original objects into the collection However, we still must be careful not to mutate the objects within the collection Since some access methods return references to the objects within the collection To be very safe our accessors should themselves return clones of the objects rather than references to the originals What is cloning?

Bonus Material: Copying and Deep vs. Shallow Copy
Java objects can be copied using the clone() method clone() is defined in class Object, so it will work for all Java classes However, you must override it for new classes to work properly It needs to know what data in the new class to copy This is somewhat tricky to do, especially for subclasses – see Employee.java for syntax clone() is already defined for Java arrays (and some other classes), so we can use it for them without overriding

clone() is typically defined to do a shallow copy of the data in an object This means that when the object is copied, objects that it refers to are NOT copied Ex: If cloning an array of StringBuilders, we get a new array but NOT new StringBuilders Show on board This can cause data sharing/aliasing that you must be aware of See Example15.java and Employee.java for example You should have also discussed this briefly in CS 0401

Generally speaking, (true) deep copying is more difficult than shallow copying We need to follow all references in the original and make copies for the clone() This could be several levels deep Ex: A Binary Search Tree The BST object has only one instance variable – a reference to the root node A shallow copy would only copy this single reference A deep copy would have to traverse the entire tree, copying each node AND copying the data in each node AND … For a deep-er copy we can use the copyNodes() which calls the copy() method that we discussed previously

These notes are intended for use by students in CS0445 at the University of Pittsburgh and no one else These notes are provided free of charge and may.

Similar presentations

Presentation on theme: "These notes are intended for use by students in CS0445 at the University of Pittsburgh and no one else These notes are provided free of charge and may."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

These notes are intended for use by students in CS0445 at the University of Pittsburgh and no one else These notes are provided free of charge and may.

Similar presentations

Presentation on theme: "These notes are intended for use by students in CS0445 at the University of Pittsburgh and no one else These notes are provided free of charge and may."— Presentation transcript:

Similar presentations

About project

Feedback