1. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. CGE7 Core Isolation Nawneet Anand

2. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Agenda What is core Isolation ? Why Isolate? Isolate From? How to Isolate? How to test? Test result. 2

3. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. What is core isolation? Core Isolation is a technique that allows a task to have the core dedicated to its needs until it exits. By this technique application can in isolation from Linux kernel scheduler,interrupts, ticks, Timers and hrtimers, workqueue. 3

4. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Why Isolate? 4 Applications with aggressive real-time constraints need to improve their worst case response time. To get better performance isolation needed. Such application can be of networking,HPC or Real time systems etc. ex- Networking: Data plane / Control Plane

5. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Why Isolate?...contd… Control Plane: Power Efficient Data Plane: High Performance –Stack present in user space to avoid context switches –Single High performance thread per cpu/core –Core must process packets at ~0 latency –So Isolate core from background kernel activities 5

6. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate: From? 6 Tasks Interrupts Ticks Timers & Hrtimers Workqueues

7. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from Tasks 7 Scheduler’s load balancer migrates tasks between CPUs –Create separate cpusets for control and data plane CPUs –Disable load balancing between them

8. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from Interrupts 8 Interrupts can land on any CPU depending on the configuration. –Most of them can be affined to a set of cores –Affine them to Control plane cores i.e boot core cpu 0.

9. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from Ticks 9 Tick: timer firing every 5-10 ms on all CPUs NO_HZ_FULL: Limits tick interval to 1 second Can get spurious ticks when hrtimer() returns without servicing any hrtimer. –To ignore this we are using -q option as command line parameter.

10. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from (hr)timers 10 Pinned & Unpinned Affined to local CPU and may be re- armed. Migrate all unpinned timers to Non- quiesced CPUs

11. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from workqueues 11 Pinned & Unpinned Affined to local CPU and may be re-armed Control them from sysfs

12. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to Isolate Boot system. Stop the IRQ balance Create Cpusets. Hotplug out/in all cpu’s we wanted to work on stress application( stess application used for demo).This will Move all tied task to other CPUs. 12

13. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to Isolate …contd.. Update cpusets for normal task movement within all CPUs(The ones which are hard bounded to a CPU wouldn’t get migrated this way. Update the CPU sets whenever we want to run a stress task on a cpu,so that the cpu in question is isolated and all its migrated using echo $taskid >/dev/cpusets/common-set- name/tasks 13

14. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to test change the mode of script Affine the Irq affinity to boot core. Check the affinity. executed the script 14

15. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to test.. Contd… Execute “ps -Leo pid,tid,psr,pcpu,comm | grep stress “ on other shell.We can see isolated core on which stress apllication executed. run “ killall stress” -it will give show on script console isolation start/end time as well previous and current total system interrupt count. 15

16. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Test Result Stress application executed on isolated core with maximum CPU utilization interrupts for last: 49 seconds test_case_id:Min-isolation 10 secs result:PASS measurement:49 units:secs 16

17. © 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Thanks 17