Download presentation
Presentation is loading. Please wait.
Published byMagnus Norton Modified over 9 years ago
1
1 Ally: OS-Transparent Packet Inspection Using Sequestered Cores Jen-Cheng Huang 1, Matteo Monchiero 2, Yoshio Turner 3, Hsien-Hsin Lee 1 1 Georgia Tech 2 Intel Labs 3 HP Labs
2
2 Deep Packet Inspection (DPI) Data Center Middle Boxes Intrusion Detection Content Insertion Traffic Classification Internet Deployment of Packet Processing Services
3
3 Problem Internet Data Center Middle Boxes Local Traffic is growing in importance… But The traffic within the data center is not inspected!
4
4 Approach “Co-locate” DPI with the server DPI appliance Server Leverage abundant CPU resources Leverage existing management interfaces on servers, e.g. HP iLO Compatible with heterogeneous architecture, e.g. on-chip accelerators
5
5 Requirements Transparency –Independent to the server’s software stack Efficiency –Low overhead packet interception Isolation –Resistant to attacks
6
6 ETTM: a scalable fault tolerant network manager. C. Dixon et al. NSDI ‘11 Related Work Transparency Hypervisor Overhead Hypervisor Vulnerability Virtualization Support for DPI deployment Hypervisor DPI VM Guest VM HW SW Virtualized Platform Processors
7
7 NIC Unprivileged Partition Multi-core processor core Privileged Partition Ally Architecture Software Stack (OS + Applications) Software Stack (DPI Application) core NIC Traffic
8
8 Outline Introduction & Motivation Architecture –Overview –Multicore Partitioning –Packet interception Evaluation Conclusions
9
9 Northbridge MMU Service Processor NIC Memory Controller Interconnect IOMMU Interrupt Unit BIOS External Network Main Memory Core Interrupt Controller MMU Core Interrupt Controller MMU Last Level Cache Management Network Baseline Architecture
10
10 Northbridge MMU Service Processor NIC Memory Controller Interconnect IOMMU Interrupt Unit BIOS External Network Main Memory Core Interrupt Controller MMU Core Interrupt Controller MMU Last Level Cache Management Network Ally Architecture Unprivileged partitionPrivileged partition
11
11 Outline Introduction & Motivation Architecture –Overview –Multicore Partitioning –Packet interception Evaluation Conclusions
12
12 Multicore Partitioning NIC Unprivileged Partition Multi-core processor core Privileged Partition Software Stack (OS + Applications) Software Stack (DPI Application) core Invisible
13
13 Core Sequestration Modify the BIOS to hide privileged core information from the OS BSP core - the first core that boots AP cores - the other cores IPI - Inter-processor interrupts OS retrieves cores information BSP AP Core Info Table Wakeup IPI Update Ally Booting Procedure: AP DPI Engine DPI core waits for IN/OUT packets Initialize …...
14
14 Memory Protection TLB TLB Miss Handler CR3 Boundary Register Page Table Range Checking Main Memory Privileged partition Unprivileged partition MMU Unprivileged Core Partition the memory into two physically contiguous regions TLB Miss TLB Fill
15
15 Outline Introduction & Motivation Architecture –Overview –Multicore Partitioning –Packet interception Evaluation Conclusions
16
16 NIC Unprivileged Partition Multi-core processor core Privileged Partition Packet Interception Software Stack (OS + Applications) Software Stack (DPI Application) core NIC Traffic
17
17 Packet Interception Virtualization of the Descriptor Queues NIC OS memory Descriptor queues replicated DPI memory Only one copy of the packet buffers Descriptor queues
18
18 Packet Interception Virtualization of the Descriptor Queues –Device independent, software independent –No copying on packet buffers Processor and NIC communication –Queue manipulation uses Memory Mapped IO (MMIO) accesses –NIC event notification uses Interrupt
19
19 MMIO redirection MMU MMU detects specific MMIO addresses MMU redirects RW to a reserved region in DPI memory MMU sends IPI to DPI core DPI memory DPI core OS core IPI R/W redirection Load/store
20
20 Ally Hardware Properties Simple extensions to existing hardware components No impact expected on critical timing paths Compatible with virtualization support (Intel VT- x/EPT, AMD SVM/NPT)
21
21 Outline Introduction & Motivation Architecture –Overview –Multicore Partitioning –Packet interception Evaluation Conclusions
22
22 Evaluation Full system emulation QEMU Core sequestration HW changes Real machine prototype Hardware –Intel Core 2 duo 2.66 GHz with 1 Gbit Intel NIC Benchmarks –Netperf –SPECweb Systems –Ally, Linux and Xen
23
23 System Configurations Queue Virtualization NIC Driver Kernel Netperf/ Specweb Snort DPI core OS coreHW SW NIC Driver Kernel Netperf/ Specweb Snort DPI core OS coreHW SW IP queue Ally Linux
24
24 System Configurations Hypervisor Dom0 Kernel Netperf/ Specweb Snort DPI core OS coreHW SW Xen DomU Kernel
25
25 Netperf CPU Usage
26
26 SPECweb CPU Usage cycles/request * 10 6
27
27 Outline Introduction & Motivation Architecture –Overview –Multicore Partitioning –Packet interception Evaluation Conclusions
28
28 Conclusions Ally: a framework for transparent deployment of packet inspection appliances Ally uses a set of simple HW/FW extensions enable reliable multicore partitioning and efficient packet inspection Ally is fully compatible with new virtualization technology as well as heterogeneous architecture
29
29 Thanks
30
30 Throughput
31
31 DPI using Network Processor
32
32 NIC Unprivileged Partition Multi-core processor core Conventional Architecture Software Stack (OS + Applications) core cores
33
33 NIC Unprivileged Partition Multi-core processor core Privileged Partition Transmission Path Software Stack (OS + Applications) Software Stack (DPI Application) core
34
34 NIC Unprivileged Partition Multi-core processor core Privileged Partition Receive Path Software Stack (OS + Applications) Software Stack (DPI Application) core
35
35 Integrated Northbridge DPI core Local APIC MMU Interface DPI core Local APIC MMU Interface OS core Local APIC MMU Interface OS core Local APIC MMU Interface Platform Controller Hub NIC Memory Controller On chip interconnect Processor IOMMUPCIe ctrl Interrupt Unit BIOS Network Main Memory DMI Ctrl OS core Local APIC MMU Interface Unprivileged partitionPrivileged partition DPI core Local APIC MMU Interface Last Level Cache IOAPIC Managemen t NIC Service Processor Management Network Privileged partition Unprivilege d partition
36
36 Integrated Northbridge DPI core Local APIC MMU Interface DPI core Local APIC MMU Interface OS core Local APIC MMU Interface OS core Local APIC MMU Interface Platform Controller Hub NIC Memory Controller On chip interconnect Processor IOMMUPCIe ctrl Interrupt Unit BIOS Network Main Memory DMI Ctrl OS core Local APIC MMU Interface Unprivileged partitionPrivileged partition DPI core Local APIC MMU Interface Last Level Cache IOAPIC Managemen t NIC Service Processor Management Network Privileged partition Unprivilege d partition
37
37 MMU Modification – Memory Protection TLB TLB Miss Handler CR3Special_reg Page Table DPI core boundary register phys_addr > special_reg ? Main Memory Privileged partition Unprivilege d partition
38
38 Memory Protection Procedure TLB TLB Miss Handler TLB miss Virtual Address CR3Special_reg Page Table DPI core boundary register phys_addr > special_reg ? Main Memory Privileged partition Unprivilege d partition
39
39 Memory Protection Procedure TLB TLB Miss Handler TLB miss Virtual Address TLB fill CR3Special_reg Page Table DPI core boundary register phys_addr > special_reg ? Main Memory Privileged partition Unprivilege d partition
40
40 NIC Unprivileged Partition Multi-core processor core Privileged Partition Memory Protection Software Stack (OS + Applications) Software Stack (DPI Application) core Invisible
41
41 Integrated Northbridge DPI core Local APIC MMU Interface DPI core Local APIC MMU Interface OS core Local APIC MMU Interface OS core Local APIC MMU Interface Platform Controller Hub NIC Memory Controller On chip interconnect Processor IOMMUPCIe ctrl Interrupt Unit BIOS Network Main Memory DMI Ctrl OS core Local APIC MMU Interface Unprivileged partitionPrivileged partition DPI core Local APIC MMU Interface Last Level Cache IOAPIC Management NIC Service Processor Management Network Privileged partition Unprivilege d partition
42
42 Integrated Northbridge DPI core Local APIC MMU Interface DPI core Local APIC MMU Interface OS core Local APIC MMU Interface OS core Local APIC MMU Interface Platform Controller Hub NIC Memory Controller On chip interconnect Processor IOMMUPCIe ctrl Interrupt Unit BIOS Network Main Memory DMI Ctrl OS core Local APIC MMU Interface Unprivileged partitionPrivileged partition DPI core Local APIC MMU Interface Last Level Cache IOAPIC Managemen t NIC Service Processor Management Network Privileged partition Unprivilege d partition
43
43 MMU Modification – MMIO Redirection TLB Redirection BitPhysical Page TLB Miss Handler Check uncacheable address map Redirection Table Physical Address Remapped Address
44
44 MMIO Redirection – TLB Miss On a TLB miss, the TLB miss handler does the page table walk TLB Redirection BitPhysical Page Virtual Address TLB miss TLB Miss Handler Page Table Lookup
45
45 MMIO Redirection – TLB Miss The TMH checks if the resulting physical address falls in an uncacheable page and hence potentially a MMIO page TLB Redirection BitPhysical Page TLB Miss Handler Physical Address Check uncacheable address map
46
46 MMIO Redirection – TLB Miss If the page is uncacheable, the TMH looks up the redirection table to check if any address in this page needs to be redirected TLB Redirection BitPhysical Page TLB Miss Handler Check uncacheable address map Redirection Table Physical Address Remapped Address Physical Address
47
47 MMIO Redirection – TLB Miss If any address in the page needs to be redirected, the TMH sets the redirection bit in addition to fill the TLB TLB Redirection BitPhysical Page TLB Miss Handler Check uncacheable address map TLB fill Redirection Table Physical Address Remapped Address
48
48 MMIO Redirection – TLB Hit On a TLB hit, if the redirection bit is set, the MMU looks up the Last Level Cache (LLC) used to cache translations in Redirection Table TLB Redirection Bit Physical Page Offset Physical Address Virtual Address LLC Physical Address Remapped Address
49
49 MMIO Redirection – TLB Hit If a translation is found, the MMU returns the translated address and sends IPI to privileged cores. TLB Redirection Bit Physical Page LLC Physical Address Remapped Address Translated Address Generate IPI Physical Address Hit
50
50 MMIO Redirection – TLB Hit If the LLC misses, then Redirection Table Lookup is performed TLB Redirection Bit Physical Page LLC Physical Address Remapped Address Redirection Table Lookup Physical Address Miss
51
51 Interrupt Unit Modification DPI core OS core Interrupt Unit NIC If Source == NIC, Redirect Interrupt
52
52 When NIC raises an interrupt, The interrupt Unit redirects the interrupt to DPI core Interrupt Redirection DPI core OS core Interrupt Unit NIC If Source == NIC, Redirect Interrupt Interrupt
53
53 After the NIC interrupt is handled, DPI core sends an IPI to OS core mimicking NIC interrupt Interrupt Redirection DPI core OS core Interrupt Unit NIC If Source == NIC, Redirect Interrupt IPI
54
54 Summary of Hardware Modifications UnitDescriptionPurpose OS-core MMU Prevent memory accesses to DPI memory from OS- core Protection Redirect MMIO accesses to DPI memory from OS- core and interrupt DPI core Packet Interception IOMMUPrevent non authorized DMA to DPI MemoryProtection IOAPICRedirect NIC interrupts to DPI-corePacket Interception All UnitsProtected configuration registersProtection
55
55 Functional Evaluation Full system emulation QEMU Validate Hardware and Firmware Changes
56
56 DPI core Usage
57
57 SPECweb Cache Misses
58
58 NIC Unprivileged Partition Multi-core processor core Privileged Partition Memory Protection Software Stack (OS + Applications) Software Stack (DPI Application) core Invisible How? Modified MMU
59
59 Challenges -Make privileged partition protected and invisible from the unprivileged partition -Core Sequestration -Memory Protection -Intercept packets efficiently -Packet Interception
60
60 Ally System NIC Linux kernel NIC Traffic Queue Virtualization NIC Driver Other Apps Snort DPI Core
61
61 Linux System NIC Linux kernel NIC Traffic IP queue NIC Driver Other Apps Snort Core
62
62 Xen System NIC Linux VM #0 NIC Traffic IP queue Hypervisor Other Apps Snort Core VM #1
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.