Data Structures Lectures: Haim Kaplan, Uri Zwick Exam: 80% Theoretical Assingnments: 10% Practical Assignments: 10% Cormen, Leiserson, Rivest and Stein Introduction to Algorithms (Second/Third Editions) (Contains only some of the material)

Heaps/Priority Queues Data Structures “In computer science, a data structure is a particular way of storing and organizing data in a computer so that it can be used efficiently.” Lists Search Trees Heaps/Priority Queues Hashing Union-Find Tries and Suffix Trees Sorting and Selection

Data Structures Lecture 1 Abstract Data Types Lists, Stacks, Queues, Deques Arrays, Linked Lists Graphs Haim Kaplan, Uri Zwick March 2018

Lists (Sequences) [a0 a1 a2 a3 … ai1 ai ai+1 … an2 an1 ] b Retrieve the item at position i Insert b at position i Delete the item at position i When items are inserted or deleted, the indices of some other items change.

Abstract Data Type (ADT) Lists (Sequences) List() – Create an empty list Length(L) – Return the length of list L Retrieve(L,i) – Return the i-th item of L Insert(L,i,b) – Insert b as the i-th item of L Delete(L,i) – Delete and return the i-th item of L ( Search(L,b) – Return the position of b in L, or −1 ) Concat(L1, L2) – Concatenate L1 and L2 Plant(L1,i, L2) – Insert L2 starting at the i-th position of L1 Split(L,i) – Split L into two lists Interesting special cases: Retrieve-First(L), Insert-First(L,b), Delete-First(L) Retrieve-Last(L), Insert-Last(L,b), Delete-Last(L)

Abstract Data Types (ADT) Stacks, Queues, Dequeues Stacks – Implement only: Insert-Last(L,b), Retrieve-Last(L), Delete-Last(L) Also known as: Push(L,b), Top(L), Pop(L) Last In First Out (LIFO) Queues – Implement only: Insert-Last(L,b), Retrieve-First(L), Delete-First(L) First In First Out (FIFO) Deques (double ended queues) – Implement only: Insert-First(L,b), Retrieve-First(L), Delete-First(L) Insert-Last(L,b), Retrieve-Last(L), Delete-Last(L)

Implementing lists using arrays We need to know the maximal length M in advance. n L array a0 a1 … an−1 * * * maxlen M length n M Retrieve(L,i) (of any element) takes O(1) time Insert-Last(L,b) and Delete-Last(L) take O(1) time Stack operations in O(1) time

Implementing lists using arrays We need to know the maximal length M in advance. n L array a0 a1 … an−1 * * * maxlen M length n M Retrieve(L,i) (of any element) takes O(1) time Insert-Last(L,b) and Delete-Last(L) take O(1) time Insert(L,i,b) and Delete(L,i) take O(n−i+1) time

Implementing lists using arrays 1 n−1 n array maxlen M length n n+1 M Delete-Last very similar

Implementing lists using arrays 1 i n−1 n array maxlen M length n n+1 M We need to move n−i items, then insert. O(n−i+1) time.

Implementing lists using circular arrays Implement queue and deque operations in O(1) time n L 1 2 array maxlen M length n M start 2 New field: start

Implementing lists using circular arrays Implement queue and deque operations in O(1) time L 1 2 M−4 M−1 array maxlen M length 7 start M−4 M Occupied region can wrap around!

Implementing lists using circular arrays Implement queue and deque operations in O(1) time n L 1 n−1 array maxlen M length n n−1 M start 1 Code of other operations similar

Arrays vs. Circular Arrays Insert/Delete Last O(1) Insert/Delete First O(n+1) Insert/Delete(i) O(n−i+1) O(min{i+1,n−i+1}) Retrieve(i) Main advantage: Constant access time. Main disadvantage: Inserting or deleting elements ‘in the middle’ is expensive.

Implementing lists using singly linked lists Lists of unbounded length Support some additional operations L first a0 a1 an1 a2 … List object List-Node object item next last length n

Insert-First with singly linked lists Insert-First(L,b) L Generate a new List-Node object B containing b first Add B to the list last Adjust L.last, if necessary length n n+1 B Increment L.length (Return B) b … item next … a0 a1 a2 an1

Insert-First with singly linked lists last length n n+1 B b … item next … a0 a1 a2 an1 Insert-Last and Delete-First – very similar, also O(1) time Unfortunately, Delete-Last requires O(n+1) time

Retrieve with singly linked lists first last length n item next … a0 a1 a2 an1 Retrieving the i-th item takes O(i+1) time

Insert with singly linked lists first last length n item next … a0 a1 a2 an1 Inserting an item into position i takes O(i+1) time

Inserting a node (After a given node) Insert-After(A,B) – Insert B after A ai−1 ai … A b B Assignments’ order is important

Deleting a node (After a given node, not the last one) Delete-After(A) – Delete the node following A A … … ai−1 ai ai+1 What happens to the node removed?

Concat(L1,L2) – Attach L2 to the end of L1 Concatenating lists Concat(L1,L2) – Attach L2 to the end of L1 L1 n+m n a0 a1 an1 a2 … L2 m b0 b1 bm1 b2 … O(1) time! (What happened to L2?)

Circular arrays vs. Linked Lists Insert/Delete-First Insert-Last O(1) Delete-Last O(n+1) Insert/Delete(i) O(min{i+1,n−i+1}) O(i+1) Retrieve(i) Concat O(min{n1,n2}+1) With linked lists we can insert or delete elements ‘in the middle’ in O(1) time

More fun with linked lists Circular singly linked lists (Circular) Doubly linked lists Doubly linked lists with a sentinel

Circular singly linked lists If we make the list circular, we don’t need first first last length n All capabilities remain the same item next … a0 a1 a2 an1

How do we implement Delete-Last(L) in O(1) time? Doubly linked lists!

(Circular) Doubly linked lists first n length prev item next … a0 a1 a2 an1 Each List-Node now has a prev field All previous benefits + Delete-Last(L) in O(1) time

Inserting a node into a Doubly Linked List Insert-After(A,B) – Insert node B after node A A ai ai+1 b B

Deleting a node from a Doubly Linked List Each node now has a prev field Delete-Node(A) – Delete node A from its list ai A … ai−1 ai+1 Note: A itself not changed! Is that good?

Circular Doubly linked lists with a sentinel … a0 a1 a2 an-1 sentinel Each node has a successor and a predecessor No special treatment of first and last elements No special treatment of empty lists In some case we can identify a list with its sentinel

Circular Doubly linked lists with a sentinel Empty list

Circular Doubly linked lists with a sentinel … a1 a2 a3 an1

Abstraction barriers User List Insert, Retrieve, Delete Search List-Node Retrieve-Node Insert-After, Delete-After, Delete-Node

Inserting and later deleting Suppose we inserted a into L. At a later stage, we want to delete a from L. With the current interface all we can do is: What we would like to do: O(n) O(n) O(1) Need to implement Search. Would take O(n) time. Did we delete the right a ?

Insert-After, Delete-Node, Next in O(1) time Modified ADT for lists The current specification does not allow us to utilize one of the main capabilities of linked lists: Insert-After, Delete-Node, Next in O(1) time We next define a new ADT in which the user is allowed to call List-Node, Insert-After, Delete-Node, Next

Lists – A modified abstraction [ a0 a1 … ai … an-1 ] Lists are now composed of List-Nodes List-Node(b) – Create a List-Node containing item b Item(B) – Return the item contained in List-Node B List() – Create an empty list Length(L) – Return the length of list L Insert(L,i,B) – Insert B as the i-th List-Node of L Retrieve(L,i) – Return the i-th List-Node of L Delete(L,i) – Delete and return the i-th List-Node of L Concat(L1, L2) – Concatenate L1 and L2

[ a0 a1 … ai … an-1 ] Lists – A modified abstraction We can now allow the following additional operations: Next(A) – Return the List-Node following A (assuming there is one) Insert-After(A,B) – Insert B after A Delete-Node(A) – Delete A from its current list These operations assume that A is contained in some list, while B is not contained in any list Note: L is not an argument of these operations!

Lists – A modified abstraction The user explicitly manipulates List-Nodes The actual implementation using linked lists remains essentially the same The length field is removed from the implementation as it is hard to keep it up to date Due to concatenations, hard to keep track of which list contains a given List-Node

Traversing a list With previous abstraction: With the new abstraction: for i ← 0 to Length(L)  1 do { a ← Retrieve(L,i) process(a) } With the new abstraction: A ← Retrieve(L,0) while (A ≠ null) { a ← Item(A) process(a) A ← Next(A) }

Pitfalls of the modified ADT … a0 a1 a2 an-1 L2 B … b0 b1 b2 bm-1 Insert-After(A,B) L2 is not a valid list now Should call Delete-Node(B) before Insert-After(A,B)

Which-List? Concatenations move List-Nodes from lists to lists Which-List(A) – return the list currently containing A Naïve implementation: scan the list from A until getting to the sentinel Much more efficient implementation possible using a Union-Find data structure.

Array-based implementation of the List/List-Node abstraction Can we implement the List/List-Node abstraction such that Retrieve will take O(1) time? L array maxlen length n M start 2 1 i b item List-Node pos list Array cells contain pointers to List-Nodes. A list node contains a pointer to the list and its position in the list. (Why is the list pointer needed?) ( How are different operations implemented?)

Implementation of lists Circular arrays Doubly Linked lists Balanced Trees Insert/Delete-First Insert/Delete-Last O(1) O(log n) Insert/Delete(i) O(i+1) Retrieve(i) Concat O(n+1) (Doubly) linked lists also support Insert-After and Delete-Node in O(1) time. This is used by later data structures.

Representing digraphs (directed graphs) Vertex A blabla name info Edge from to B out D B A C E F

[ , , … ] Adjacency lists representation of digraphs Graph vertices D B A C E F List of vertices [ , , … ] A B C out [ , … ] , List of outgoing edges of A [ , … ] , List of outgoing edges of B Do we need the from field? Keep a list of vertices. For each vertex keep a list of its outgoing edges. How are the lists implemented?

Should List-Nodes contain items, or point to items? Suppose we use linked lists to represent adjacency lists. Option 1: List-Nodes point to Edges … e2 e1 e0 ek1 sentinel Edge from to info Disadvantage: Another level of indirection Advantage: An item may belong to several lists Are the dotted pointers useful?

Should List-Nodes contain items, or point to items? Suppose we use linked lists to represent adjacency lists. Option 2: List-Nodes contain Edges Edge from to info prev next … sentinel Edge(List-Node) Less pointers  Faster and more compact List-Node<Edge> An item can be in only one list

Adjacency lists representation of digraphs B A C E F Using a linked-list representation of adjacency lists we can in O(1) time: 1 2 3 Move from an edge to the edge following it or preceding it in the corresponding adjacency list. Delete or insert edges before or after a given edge. We cannot efficiently traverse incoming edges of a vertex. Adjacency lists define an (arbitrary) order on the outgoing edges of each vertex.

Incoming and outgoing adjacency lists For each vertex keep both an incoming and an outgoing adjacency lists D B A C E F Each Edge now contained in two Lists Each Edge points to the two List-Nodes containing it Edge from to info in-node out-node Vertex A blabla name info in out

Incoming and outgoing adjacency lists Option 1: List-Nodes point to Edges … e2 e1 e0 ek1 sentinel Edge A B from to info in-node out-node Outgoing edges of A … f2 f1 f0 fm1 sentinel Incoming edges of B

Incoming and outgoing adjacency lists Option 2: An Edge functions as two List-Nodes Edge A B from to info in-prev in-next out-prev out-next Disadvantage: Cannot use List/List-Nodes as an ADT. Need to copy their implementation, twice.

Manipulating graphs Suppose we use both incoming and outgoing adjacency lists, with List-Nodes pointing to edges (option 1). Generating an empty graph: The out-edge following E: G  Graph() Item(Next(E.out-node)) Adding two vertices A and B: Suppose E and F both emanate from A. Move F immediately after E: A  Vertex() Insert-First(G.vertices,List-Node(A)) B  Vertex() Insert-First(G.vertices,List-Node(B)) Delete-Node(F.out-node) Insert-After(E.out-node,F.out-node) Delete edge E from the graph: Adding an edge AB: Delete-Node(E.in-node) Delete-Node(E.out-node) EEdge(A,B) E.in-node  List-Node(E) E.out-node  List-Node(E) Insert-First(B.in,E.in-node) Insert-First(A.out,E.out-node) Delete vertex A and all edges adjacent to it: …