Data Structures for Shaping and Scheduling Heap Calendar Queue Timing Wheel ECE 1545

Data Structures Data structures used to perform scheduling and shaping: Priority queues Calendar queues Timing wheel

Priority Queue A priority queue is an abstract data type that stores a collection of data with a priority Priority queue operations insert: insert an element with a priority deleteMin: remove the element with the lowest priority Property of priority queue: If two elements x and y are in the queue, if x has a lower priority than y, x will be removed before y

Implementing a Priority Queue: Candidates Linked list O(N) worst-case time on either insert() or deleteMin() Binary Search Tree O(log N) average time on insert() and deleteMin() Overkill: all elements are sorted However, we only need the minimum element Heap O(log N) worst case for both insertion and deleteMin operation

Heap A heap is a binary tree which satisfies the heap property: “The key of every node is less than or equal to the key of any of its children.” Properties: The smallest element in a heap is the root No conclusion can be drawn about the order of children The tree is completely filled on all levels except possibly the lowest root 3 4 7 5 6

Vector Representation Use a vector A to store a heap Store elements in level-order Parent (k) : ⌊ k/2 ⌋ Left (k) : 2k Right (k): 2k + 1 Heap property: A[Parent(k)] ≤ A[k] root R l r ll lr rl rr 7 6 5 4 3 2 1 A: R l r ll lr rl rr

insert(x) To insert an element with value x: Create a new leaf node If heap property is violated, swap node with parent node Repeat until parent node is less than x

insert(14) 14 14 14 Example from slides by A. Srinivasan (Florida State)

deleteMin() Replace root with the last leaf (last vector element) This maintains the complete binary tree property but may violate the partially ordered tree property Start at root: If heap property is violated swap position with smaller child 31 31

deleteMin() 31 31 Example from slides by A. Srinivasan (Florida State) 31

Calendar Queue A calendar queue is a priority queue with where the average enqueue and dequeue complexity is O(1) Implemented as an array of sorted linked lists Element of the array is called a ”bucket” Each bucket is associated with a time interval (“width”) Elements of linked list are sorted buckets linked lists current time pointer …

Calendar Queue … Inspired by a desk calendar: one page (bucket) per day of year items of same day are listed in sorted order listed items can be for subsequent years items not in current year are ignored buckets linked lists current time pointer March 18 3/18/2019, 3pm 3/18/2019, 6:30pm 3/18/2020 8am 2019 …

Calendar Queue Calendar queue with n buckets, where each bucket has width w Value x is added to bucket x/w mod n Important: Number and width of buckets must be adjusted Buckets should not have many items in list otherwise enqueue into sorted list is too long Not too many buckets should be empty  otherwise dequeue must search too many buckets Bucket width should not be too large  otherwise data is clustered in few buckets and most other buckets are empty Bucket width should not be too small  otherwise most data is not in “current year”

Calendar Queue: Adjusting number of buckets Target: One item for each bucket Rules of thumb: Double the number of buckets when number of stored entries is twice the number of buckets Halve the number of buckets when number of stored entries is half the number of buckets When bucket number is adjusted, copy items to a new calendar queue

Calendar Queue: Adjusting the bucket width Target: Set Bucket width equal to the average separation of adjacent elements  minimizes enqueue and dequeue times Rules of thumb: Recompute bucket width when copying to a new calendar queue Select samples around place of current time pointer and estimate average separation of adjacent elements Set bucket width to estimate of average separation

Calendar Queue: Tightly clustered items 1000 items, 512 buckets 500 items uniformly in [0, 0.1] 500 items uniformly in [5.6, 5.7] After removing elements in [0,0.01], current time pointer has to cycle through 54 “years” for next items Solution: After running through an “empty year”, make global search for smallest item bucket width = 0.002 buckets linked lists current time pointer [0, 0.002) 5.6 [0.002, 0.004) 0.002 5.6002 [0.004, 0.006) 0.004 5.6004 [0.006, 0.008) 0.006 5.6006 … [1.022, 1.024)

Calendar Queue: Evaluation Evaluation from 1988 (on 5 Mhz PC) Exponential separation between adjacent elements Comparison: Linear linked list Splay tree Supports claim of constant enqueue and dequeue times

Calendar Queue: Evaluation Different distributions of separation:

Calendar Queue: Applications Main application for calendar queue is the management of events in discrete event simulations Suitable to compute timers (for timeout values) Suitable for shaping functions that compute the release time (eligibility time) of packets Less suitable for a sorted scheduler queues (e.g., EDF, WFQ) Due to potential search for smallest element

Timing Wheel O(1) data structure for managing timers Inspired by timing wheel in logic simulators

Simple Timing Wheel Assumes that timers are set to at most MAX value into the future (Here: MAX=8), i.e., within a rotation from the current position A cursor in the timing wheel moves one location every time unit A list for each location If cursor is at position i and if timer is set for i+j, add timer to position (i+j) mod MAX 1 7 2 6 3 5 4 Enqueue and dequeue is O(1) Drawback: required wheel size can be large

Hashed Timing Wheel Works for arbitrary values of the timer Record the number of remaining rounds with the stored entry If cursor is at position i and if timer is set for i+j, add timer to position (i+j) mod MAX with remaining rounds (i+j) div MAX Lists for a position can be sorted or unsorted # of rounds remaining 4 1 7 2 6 3 1 2 5 4 2 1 In the worst-case, enqueue is O(n) with sorted lists If # entries is less than wheel size, average enqueue should be O(1)

Hierarchical Timing Wheel 10:25:30 Seconds pointer advanced per clock tick If seconds pointer wraps, move the minutes pointer If minutes pointer wraps, move the hour pointer Example: Current time: 10h 25m 30s Timer value: 50m 50s Timeout at: 11h 15m 20s Add entry for 11h, also store minutes and seconds hours wheel 1 23 2 … … 11 10 20 15 1 minutes wheel 59 2 … 3 25 … 1 seconds wheel 59 2 … 3 … 4

Hierarchical Timing Wheel 11:00:00 At 11h 0m 0s, remove all entries from 11h slot and move to minutes wheel, into position of minute value In minutes wheel, store the seconds hours wheel 1 23 2 … … 11 10 20 15 1 minutes wheel 59 2 … 3 15 … 20 1 seconds wheel 59 2 … 3 … 4

Hierarchical Timing Wheel 11:15:00 At 11h 15m 0s, remove all entries from 15m slot and move to seconds wheel, into position of seconds value At 11h 15m 20s, the seconds wheel moves to 20s and finds the entry hour wheel 1 23 2 … … 11 10 1 minute wheel 59 2 … 3 15 … 20 1 seconds wheel 59 2 … 3 20 4

Timing Wheels: Summary Application areas for timing wheels similar to calendar queues: Timers Simulations Shapers Hashed timing wheel is essentially a calendar queue Main difference of timing wheel and calendar queue: Timing wheel does not resize