Bolt : Faster Reconfiguration in Operating Systems Sankaralingam Panneerselvam Michael M. Swift Nam Sung Kim University of Wisconsin, Madison, WI ATC 2015

2 Contents Introduction Motivation Bolt Evaluation

3 Introduction OS Os used a fixed set of CPU, Processor, resources Dynamic reconfiguration is current computing landscape Benefits of Processor Reconfiguration Disaggregated Server Architectures Energy SavingsPerformance Inflation

4 Introduction Support for Reconfiguration Hotplug mechanism in Linux Designed as a - Reliability mechanism to remove faulty cores - Testing mechanism to vary number of cores Latency in the order of tens or hundreds of milliseconds in current Linux Bolt Faster reconfiguration method 20x faster than Hotplug

5 Motivation Process scaling Changed core set dynamically Energy proportionality - Amount of work is determined energy consumption - Low and sleep state is for energy consumption Heterogeneity - Heterogeneity is supported many processors - Os can reconfiguration processor considered applications characteristic VM Scaling - Calculated VM core in Virtual machine - IBM support DLPAR can add and remove Virtural core with interface

6 Motivation Hot plug Os subsystems subscribe for CPU reconfiguration events Notifications sent during reconfiguration event - Addition-Online/Removal-Offline - Thread migration - Watchdog enable/disable Drawback - Hotplug event can be prevented by disabling preemption or interrupts on any CPU - Another machine or core can be stoped by Hot plug during setting new a set of cores - Synchorous all time

7 Bolt Based on Hotplug but can be used as a replacement Classify subsystems notification handler into - Critical : Need to be executed immediately for correctness(E.g. Migrating threads) - Non-Critical : Immediate action is not necessary (E.g. Freeing a memory structure) Critical session - Critical operations are handled synchronously CategoryExamples State Migration Threads in runqueue, softirqs in per-cpu queue Hardware Management Microcode update, updating MTRR registers Bitmask updates Global structures like cpu_online_mask Re-organization Scheduler domain structures

8 Bolt Non-Critical operations Almost 30 out of 50 callbacks were identified as non-critical operations Bolt removes them from critical path but retaining the interface CategoryHow handled? Software structuresMemory cleanup deferred to a master thread Sysfs file access is checked during open Thread operationsParking/Unparking made asynchronous Thread re-use without destroying them

9 Bolt Bulk Interface Bulk interface support to allow scaling multiple CPUs API can add or remove a index of single CPU In Bolt Aggregation : avoid redundant execution Parallel Execution : Dying message is handled in parallel by all cores in cpu mask

10 Bolt

11 Evaluation Enviroment X86 : 4-core Intel i5-2500k(sandbridge) ARM : big.LITTLE Exynos 5410(odroid development board) Experiments : Latency measured for Hotplug operations

12 Evaluation Synchrous of bitmask Parking thread End to end latency

13 Evaluation

14 Evaluation

15 Conclusion Hotplug is currently used as a reconfiguration mechanism Bolt reduces reconfiguration latency by separating critical from non- critical operations Bolt’s principles can be applied to any reconfiguration mechanism