Micro-Segmentation Support For Vmware vDS Part 2

Objectives and Assumptions Objectives: After completing this module you will: Understand what is new with DVS Micro-segmentation. Know why the feature was developed, it’s and benefits. Comprehend the various implementation scenarios. Assumptions: Students must be knowledgeable in Micro-segmentation and Intra-EPG Isolation feature of ACI.

Agenda Overview Architecture Packet Flow Configuration Implementation Forwarding Scenarios Troubleshooting and Debugging Demo Questions? Appendix

Acronym Decoder VDS/DVS – Virtual Distributed Switch uSeg – Micro-Segmentation VLAN – Virtual LAN PVLAN – Private VLAN EPG – End Point Group BD - Bridge Domain VMM – Virtual Machine Manager ACI – Application Centric Infrastructure APIC – Application Policy Infrastructure Controller

Forwarding Scenarios

Base EPG With ‘Allow Micro-Segmentation’ + uSeg EPG

Base EPG With ‘Allow Micro-Segmentation & Isolation Enforced + Useg EPG With Isolation Enforced

Troubleshooting

Troubleshooting CLI’s and Debugs EPG verification: show vlan vsh_lc -c "show system internal eltmc info vlan br” vsh_lc -c "show system internal eltmc info vlan " | egrep "primary_encap|access_encap|sclass|proxy_arp" End Point verification: show system internal epm vlan all show system internal epm vlan detail show system internal epm endpoint mac show system internal epm endpoint ip vsh_lc -c "show system internal epm endpoint mac " vsh_lc -c "show system internal epm endpoint ip " show platform internal hal ep l3 ip /32 show platform internal hal ep l2 mac show platform internal hal object ep l3 ip /32 show platform internal hal object ep l2 mac Contract verification: show zoning-rule show system internal policy-mgr stats

Toplogy Vs Vp PVlanMap DVS Vp Vs Base1Base2 (intra EPG Deny) Mac Useg Regular EPG PcTag=49160 PcTag=49159 PcTag=16390 PcTag=32771 Regular ARP, L2/L3 Proxy ARP, /32 routing Intra-EPG Deny

Verify Configuration For Deployed EPG’s Only one FD VLAN for all base EPG’s / BD, here FD vlan 8 for base_epg1 and base_epg2 and useg epg’s

Check /mit/sys For l2MacCktEp Depolyment Only on SB based TOR, operSt will be “up”. On others, it will be shown as “unsupported” In general it is good to check /mit/sys for all the concrete objects

Verify Configured l2MacCktEp In BD (EPM/EPMC) MAC Ckt will be created for all EP’s in Base EPG’s (EPG with allow useg enabled) and Useg EPG’s with MAC and VM attribute

Verify Configured EP Details From EPM

Verify Configured EP Details From EPMC

Verify Configured EP Details From HAL Objects

Verify Configured EP Details From HAL Hardware Programming

Verify Contracts / Zoning-Rules

Troubleshooting – Case Studies Port-group not created for base EPG: Check for faults in: Tenant -> EPG VM Networking -> VMM Domain Verify vmmEpPD and compEpPD are created and has encap allocated Verify DVS has PVLAN mapping configured VM doesn’t move to useg EPG (is not displayed in client endpoint table) Check for faults in: Tenant -> useg EPG Verify Base and Useg EPG are associated to the same BD and VMM domain Verify compEpPD exists under compCtrlr for useg EPG Verify user-configured attributes under fvEpCP Verify compRsDlPol under compVNic is pointing to port-group for base EPG Verify compRsUsegEpPD under compVNic Verify fvDyMacAttrDef Verify l2MacCktEp has correct class-id/pcTag Verify l2RsPathDomAtt has correct PVLAN

Attribute Preference AttributePrecedence IP Sets1 MAC Sets2 VNIC (DN)3 VM (ID)4 VM Name5 Hypervisor6 Domain (DVS)7 Datacenter8 Custom Attribute9 Guest OS10 DVS port-group11

Operator Preference OperatorPrecedence Equals1 Contains2 Starts With3 Ends With4

Proxy ARP Scenarios

ARP Request From Isolated EP(l1) to Isolated EP2 (l3): ARP Ucast Mode, EP2 Unknown Isolated EPG, Vi (pvlan) Regular EPG, Vni BD:Vb ARP Req (I1, I3) I1 I2I3 NI1 Glean (I1, I3) Glean ARP req (SVI, I3) L1 L2 L3 I1 wants to talk to I3 1=> I1 sends ARP req for I3 2=> I3 not known at L1, Pkt sent to Spine Proxy 3=> Spine does not know I3, initiates glean to all Leafs 4=> L1, L2, L3 sends Glean ARP req to I3 with Source as SVI IP and SVI Mac 5=>I3 sends ARP Reply to SVI MAC Now L2 Knows I3 I1 still does not still get Mac resolution for I3 So it keeps sending ARP requests for I3 ARP reply from I3 to SVI MAC

ARP Request From Isolated EP1(l1) to Isolated EP2 (l3): ARP Ucast Mode, EP2 Known Isolated EPG, Vi (pvlan) Regular EPG, Vni BD:Vb ARP Req (I1, I3) I1 I2I3 NI L1 L2 L3 I1 wants to talk to I3, I3 is known in Fabric 1=> I1 sends ARP req for I3 2=> I3 not known at L1, Pkt sent to Spine Proxy. Pkt is marked with TC=7 indicating src is Isolated EPG 3=> Spine sends packet to L2 as I3 is behind L2 4=> TC=7, I3 in Isolated Vlan => Pkt punted to CPU SUP generates Proxy ARP response for I3 with SVI Mac to I1 Similarly I3 resolves I1 in 2-step process Now I3 and I1 can communicate via Leaf as Router 1 Proxy ARP Resp (I3, SVI MAC,I1, I1 MAC)

ARP Request From Isolated EP1(l1) to Non Isolated EP2 (l1): ARP Ucast Mode, EP2 Unknown Isolated EPG, Vi (pvlan) Regular EPG, Vni BD:Vb ARP Req (I1, NI1) I1 I2I3 NI1 Glean (I1, NI1) Glean ARP req (SVI, NI1) L1 L2 L3 I1 wants to talk to NI1 1=> I1 sends ARP req for NI1 2=> NI1 not known at L1, Pkt sent to Spine Proxy 3=> Spine does not know NI1, initiates glean to all Leafs 4=> L1, L2, L3 sends Glean ARP req to NI1 with Source as SVI IP and SVI Mac 5=>NI1 sends ARP Reply to SVI MAC Now L3 Knows NI1 I1 does not still get Mac resolution for NI1 So it keeps sending ARP requests for NI1 ARP reply from NI1 to SVI MAC

ARP Request From Isolated EP(l1) to Non Isolated EP2 (l1): ARP Ucast Mode, EP2 Known Isolated EPG, Vi (pvlan) Regular EPG, Vni BD:Vb ARP Req (I1, NI1) I1 I2I3 NI1 Original ARP req (I1, NI1) L1 L2 L3 I1 wants to talk to NI1, NI1 known in L3 1=> I1 sends ARP req for NI1 2=> NI1 not known at L1, Pkt sent to Spine Proxy 3=> Spine sends the packet to L3 4=> L3 sends original ARP request to NI1 5=>NI1 sends ARP Reply to I1, I1-MAC Now I1 and NI1 know each other’s MAC and can communicate in Layer2 ARP reply from NI1 to I1-MAC

ARP Request From Isolated EP(l1) to Isolated EP2 (l3): ARP Flood Mode, EP2 Unknown Isolated EPG, Vi (pvlan) Regular EPG, Vni BD:Vb ARP Req (I1, I3) I1 I2I3 NI1 Glean (I1, I3) Glean ARP req (SVI, I3) L1 L2 L3 I1 wants to talk to I3 1=> I1 sends ARP req for I3 2=> I3 not known at L1, Pkt is flooded locally in Vni and to Spine with TC=7 3=> Spine floods the packet to all leafs where BD is present. 4=> Also spine sends glean packet to all Leafs 5=> Original flooded ARP request sent out of leafs only in Vni.. In Vi, it is dropped. 6=> L1, L2, L3 sends Glean ARP req to I3 with Source as SVI IP and SVI Mac 7=>I3 sends ARP Reply to SVI MAC Now leaf L2 Knows I3 I1 does not still get Mac resolution for I3 So it keeps sending ARP requests for I3 ARP reply from I3 to SVI MAC 3 4 ARP Req (I1, I3) 5

ARP Request From Isolated EP(l1) to Isolated EP2 (l3): ARP Flood Mode, EP2 Known Isolated EPG, Vi (pvlan) Regular EPG, Vni BD:Vb ARP Req (I1, I3) I1 I2I3 NI1 Glean ARP req (SVI, I3) L1 L2 L3 I1 wants to talk to I3 1=> I1 sends ARP req for I3 2=> I3 not known at L1, Pkt is flooded locally in Vni and to Spine with TC=7 3=> Spine floods the packet to all leafs where BD is present. 4=> Original flooded ARP request sent out of leafs only in Vni.. In Vi, it is dropped. Here copy the flooded packet to CPU and then generate Glean 5=> L1, L3 sends Glean ARP req to I3 with Source as SVI IP and SVI Mac 6=>Leaf L2 has I3 learnt locally so it will send Proxy ARP Reply for I3 to I1 with SVI MAC Now for I3, I1 has mac= SVI MAC, similarly I3 will resolve I1 with SVI MAC and I1 and I3 will communicate Via Leaf as router 3 ARP Req (I1, I3) 4 ARP Req (I1, I3) (ivxlan) Proxy ARP Resp (I3, SVI MAC,I1, I1 MAC) 7

ARP Request From Isolated EP(l1) to Non Isolated EP2 (Nl3): ARP Flood Mode, EP2 Unknown Or Known Isolated EPG, Vi (pvlan) Regular EPG, Vni BD:Vb ARP Req (I1, NI1) I1 I2I3 NI1 Glean (I1, NI1) Glean ARP req (SVI, NI1) L1 L2 L3 I1 wants to talk to NI1 1=> I1 sends ARP req for NI1 2=> NI1 not known at L1, Pkt is flooded locally in Vni and to Spine with TC=7 3=> Spine floods the packet to all leafs where BD is present. 4=> Also spine sends glean packet to all Leafs (if EP unknown) 5=> Original flooded ARP request sent out of leafs only in Vni.. In Vi, it is dropped. 6=> L1, L2, L3 sends Glean ARP req to NI1 with Source as SVI IP and SVI Mac (If EP unknown) 7=>NI1 sends ARP Reply to I1 (response to 5), I1- MAC with NI1-MAC. This response is L2 switched and goes to I1 Now I1 knows NI1-MAC and they can communicate 3 4 ARP Req (I1, NI1) 5 ARP reply from NI1 to I1-MAC ARP Req (I1, NI1) (ivxlan)

ARP Request From Non Isolated EP1(Nl1) to Isolated EP2 (l1): ARP request from Non-Isolated EP to Isolated EP is treated as if between two regular EPGs, just like today in all the modes