© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. CGE7 Core Isolation Nawneet Anand

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Agenda What is core Isolation ? Why Isolate? Isolate From? How to Isolate? How to test? Test result. 2

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. What is core isolation? Core Isolation is a technique that allows a task to have the core dedicated to its needs until it exits. By this technique application can in isolation from Linux kernel scheduler,interrupts, ticks, Timers and hrtimers, workqueue. 3

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Why Isolate? 4 Applications with aggressive real-time constraints need to improve their worst case response time. To get better performance isolation needed. Such application can be of networking,HPC or Real time systems etc. ex- Networking: Data plane / Control Plane

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Why Isolate?...contd… Control Plane: Power Efficient Data Plane: High Performance –Stack present in user space to avoid context switches –Single High performance thread per cpu/core –Core must process packets at ~0 latency –So Isolate core from background kernel activities 5

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate: From? 6 Tasks Interrupts Ticks Timers & Hrtimers Workqueues

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from Tasks 7 Scheduler’s load balancer migrates tasks between CPUs –Create separate cpusets for control and data plane CPUs –Disable load balancing between them

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from Interrupts 8 Interrupts can land on any CPU depending on the configuration. –Most of them can be affined to a set of cores –Affine them to Control plane cores i.e boot core cpu 0.

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from Ticks 9 Tick: timer firing every 5-10 ms on all CPUs NO_HZ_FULL: Limits tick interval to 1 second Can get spurious ticks when hrtimer() returns without servicing any hrtimer. –To ignore this we are using -q option as command line parameter.

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from (hr)timers 10 Pinned & Unpinned Affined to local CPU and may be re- armed. Migrate all unpinned timers to Non- quiesced CPUs

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Isolate from workqueues 11 Pinned & Unpinned Affined to local CPU and may be re-armed Control them from sysfs

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to Isolate Boot system. Stop the IRQ balance Create Cpusets. Hotplug out/in all cpu’s we wanted to work on stress application( stess application used for demo).This will Move all tied task to other CPUs. 12

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to Isolate …contd.. Update cpusets for normal task movement within all CPUs(The ones which are hard bounded to a CPU wouldn’t get migrated this way. Update the CPU sets whenever we want to run a stress task on a cpu,so that the cpu in question is isolated and all its migrated using echo $taskid >/dev/cpusets/common-set- name/tasks 13

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to test change the mode of script Affine the Irq affinity to boot core. Check the affinity. executed the script 14

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. How to test.. Contd… Execute “ps -Leo pid,tid,psr,pcpu,comm | grep stress “ on other shell.We can see isolated core on which stress apllication executed. run “ killall stress” -it will give show on script console isolation start/end time as well previous and current total system interrupt count. 15

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Test Result Stress application executed on isolated core with maximum CPU utilization interrupts for last: 49 seconds test_case_id:Min-isolation 10 secs result:PASS measurement:49 units:secs 16

© 2013 MontaVista Software, LLC. MontaVista Confidential and Proprietary. Thanks 17