1. Bolt : Faster Reconfiguration in Operating Systems Sankaralingam Panneerselvam Michael M. Swift Nam Sung Kim University of Wisconsin, Madison, WI ATC 2015

2. 2 Contents Introduction Motivation Bolt Evaluation

3. 3 Introduction OS Os used a fixed set of CPU, Processor, resources Dynamic reconfiguration is current computing landscape Benefits of Processor Reconfiguration Disaggregated Server Architectures Energy SavingsPerformance Inflation

4. 4 Introduction Support for Reconfiguration Hotplug mechanism in Linux Designed as a - Reliability mechanism to remove faulty cores - Testing mechanism to vary number of cores Latency in the order of tens or hundreds of milliseconds in current Linux Bolt Faster reconfiguration method 20x faster than Hotplug

5. 5 Motivation Process scaling Changed core set dynamically Energy proportionality - Amount of work is determined energy consumption - Low and sleep state is for energy consumption Heterogeneity - Heterogeneity is supported many processors - Os can reconfiguration processor considered applications characteristic VM Scaling - Calculated VM core in Virtual machine - IBM support DLPAR can add and remove Virtural core with interface

6. 6 Motivation Hot plug Os subsystems subscribe for CPU reconfiguration events Notifications sent during reconfiguration event - Addition-Online/Removal-Offline - Thread migration - Watchdog enable/disable Drawback - Hotplug event can be prevented by disabling preemption or interrupts on any CPU - Another machine or core can be stoped by Hot plug during setting new a set of cores - Synchorous all time

7. 7 Bolt Based on Hotplug but can be used as a replacement Classify subsystems notification handler into - Critical : Need to be executed immediately for correctness(E.g. Migrating threads) - Non-Critical : Immediate action is not necessary (E.g. Freeing a memory structure) Critical session - Critical operations are handled synchronously CategoryExamples State Migration Threads in runqueue, softirqs in per-cpu queue Hardware Management Microcode update, updating MTRR registers Bitmask updates Global structures like cpu_online_mask Re-organization Scheduler domain structures

8. 8 Bolt Non-Critical operations Almost 30 out of 50 callbacks were identified as non-critical operations Bolt removes them from critical path but retaining the interface CategoryHow handled? Software structuresMemory cleanup deferred to a master thread Sysfs file access is checked during open Thread operationsParking/Unparking made asynchronous Thread re-use without destroying them

9. 9 Bolt Bulk Interface Bulk interface support to allow scaling multiple CPUs API can add or remove a index of single CPU In Bolt Aggregation : avoid redundant execution Parallel Execution : Dying message is handled in parallel by all cores in cpu mask

10. 10 Bolt

11. 11 Evaluation Enviroment X86 : 4-core Intel i5-2500k(sandbridge) ARM : big.LITTLE Exynos 5410(odroid development board) Experiments : Latency measured for Hotplug operations

12. 12 Evaluation Synchrous of bitmask Parking thread End to end latency

13. 13 Evaluation

14. 14 Evaluation

15. 15 Conclusion Hotplug is currently used as a reconfiguration mechanism Bolt reduces reconfiguration latency by separating critical from non- critical operations Bolt’s principles can be applied to any reconfiguration mechanism