Microsoft 2016 2/25/2018 11:33 AM BRK4007 Troubleshoot media flows in Skype for Business across online, server and hybrid Thomas Binder Senior Program Manager © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda The challenge The solution In action Call flows

About me tbinder @microsoft.com Me Vienna, Austria Since 2007 My daughter Product Group Readiness

About this session Scope What you should already know Limited to media scenarios Server, Service, Hybrid What you should already know Basic understanding of SIP and RTP Basic understanding of the Skype for Business server roles Basic understanding of a typical Skype for Business topology

Terms & Acronyms Server Service Candidate ICE STUN TURN Lync Server 2010, Lync Server 2013, Skype for Business Server 2015 Service Skype for Business Online Candidate Possible combination of IP address and port for media channel ICE Interactive Connectivity Establishment STUN Simple Traversal of UDP through NAT Session Traversal Utilities for NAT TURN Traversal Using Relay NAT

2/25/2018 11:33 AM The challenge © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

The Challenge (Server) TechReady 18 2/25/2018 The Challenge (Server) Signaling Media NAT NAT Alice Bob Corporate firewall SIP Proxy Corporate firewall Registrar Charlie Dan © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

The Challenge (Service) TechReady 18 2/25/2018 The Challenge (Service) Signaling Media NAT NAT Alice Bob Corporate firewall Charlie Dan © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Challenge 1: NAT Network Address Translation Function Tradeoff Translates one or more internal addresses to one external address Allows connections from private network Blocks connection from public networks Tradeoff Security vs. usability Blocks unwanted traffic Might also block wanted traffic NAT Alice

Challenge 2: Corporate Firewalls Though more scrutinized, goals are similar Sharing of IP addresses Controlling data traffic from the internet Two firewalls isolate via perimeter network external internal Outer Firewall Inner Firewall

Signaling Solution SIP Proxy Reachable: on the Internet Proxies all SIP traffic external SIP Proxy Registrar Outer Firewall Inner Firewall

Putting it together Signaling uses SIP Proxy Media flows over separate channel Pre-ICE endpoints uses local IPs & ports No media can be sent between (a) and (w) SIP Proxy external internal a w Outer Firewall Inner Firewall NAT

2/25/2018 11:33 AM The solution © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Solution: ICE, STUN, TURN Add a AV Edge Server STUN reflects NAT addresses (b) and (e) TURN relays media packets (c) (d) (x) (y) ICE exchanges candidates and determines optimal media path All three protocols based IETF standards/drafts SIP Proxy external internal a b c w STUN/ TURN Server e d x y Outer Firewall Inner Firewall NAT

Who uses ICE? ICE endpoints Terminates media Edge Server Clients, servers, service Terminates media Audio Video Desktop/Application Sharing 1:1 File Transfer (Not: PowerPoint sharing) Exception: Video Interop Server Edge Server Provides STUN and TURN Does not terminate any media Is not an ICE endpoint

2/25/2018 11:33 AM In action © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Five phases of ICE During sign-in When establishing a call Requesting token from Media Relay Authentication Service (MRAS) When establishing a call Candidate Discovery Candidate Exchange Connectivity Checks Candidate Promotion

Credentials for Remote Client SIP Register Access Edge 200 OK ms-user-logon-data: RemoteUser <mrasUri>sip:Mras.contoso.com SIP Service <identity>tbinder@...</identity> 200 OK <credentials> <mediaRelayList> Service MRAS Front End Server 200 OK Endpoint AV Edge Outer Firewall Inner Firewall

Credentials for anonymous user Outer Firewall Inner Firewall SIP Invite Access Edge 200 OK <Credentials> <mediaRelayList> Service MRAS Front End Server 200 OK Endpoint AV Edge

Demo Log Analysis: acquiring MRAS credentials

Address Discovery Audio/Video/Video Based Screen Sharing UDP TCP a NIC 1 NAT/Firewall c default a MRAS b c candidates allocate UDP b c d e allocate TCP d e local remote Endpoint AV Edge

Address Discovery RDP based screen sharing/File Transfer UDP TCP a NIC 1 NAT/Firewall c default a MRAS b c candidates allocate TCP b c local remote Endpoint AV Edge

Address Exchange a b NAT/Firewall w v NAT/Firewall d x AV Edge AV Edge NIC NIC d x SIP INVITE c :: a, b, c, d, e c default y c default y 183 Session progress y :: v, w, x, y, z a v a v 200 OK y :: v, w, x, y, z b w b w AV Edge AV Edge c candidates x c candidates x c y d y d y e z e z e z local remote local remote Endpoint Endpoint SIP

Demo Log Analysis: Candidates

Connectivity Checks Determine all possible UDP and TCP port pairings Edge Server can bridge between IPv4 and IPv6 STUN packets sent between port pairs in order STUN packet response indicates connectivity Stop checks when candidate pair has bi-directional connectivity

Candidate Promotion Select highest order candidate with validated connectivity IPv4 before IPv6 Direct before relay UDP before TCP Send SIP invite, indicating only candidate is in SDP 200 OK also contains only one candidate in SDP RTP and RTCP will each gave a candidate Media is on optimal, validated path

Demo Log Analysis: Final Candidates

2/25/2018 11:33 AM Call flows © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Server: Topology Outer Firewall Inner Firewall NAT External 1 Internal 1 UDP 3478 TCP 443 UDP/TCP 50,000 . UDP/TCP 59,999 Internal 2 External 2 NAT AV Edge

Server: Inside/Inside Outer Firewall Inner Firewall w1 Internal 1 UDP 3478 TCP 443 w1 UDP/TCP 50,000 . UDP/TCP 59,999 w1 Internal 2 w2 w2 w2 AV Edge

Server: Inside/Outside Outer Firewall Inner Firewall External 1 w1 Internal 1 h1 h1 UDP 3478 TCP 443 w1 UDP/TCP 50,000 . UDP/TCP 59,999 w1 w2 w2 AV Edge

Server: Inside/Outside Outer Firewall Inner Firewall External 1 h1 h1 UDP 3478 TCP 443 w1 UDP/TCP 50,000 . UDP/TCP 59,999 w1 External 2 h2 h2 w2 w2 NAT AV Edge

Service: Topology NAT Firewall External 1 UDP 3478 TCP 443 . UDP/TCP 59,999 External 2 NAT AV Edge

Service: “Outside/Outside” Firewall External 1 h1 h1 UDP 3478 TCP 443 w1 UDP/TCP 50,000 . UDP/TCP 59,999 w1 External 2 h2 h2 w2 w2 NAT AV Edge

Edge to Edge connection Inner Firewall Outer Firewall Outer Firewall Inner Firewall w1 Endpoint 1 Endpoint 2 w2 UDP 3478 UDP 3478 TCP 443 TCP 443 w1 UDP/TCP 50,000 . UDP/TCP 59,999 w2 UDP/TCP 50,000 . UDP/TCP 59,999 w2 AV Edge AV Edge

Server: 50,000 requirements - Minimum Lync Server 2010 & 2013, Skype for Business Server 2015 Port requirements between AV Edge Server and Internet Requires “50,000-59,999 TCP outbound” Workload independent All workloads use same port ranges Source IP Destination IP A/V Edge service interface Any Source Port Destination Port UDP 3478 TCP 50,000-59,999 TCP 443 Any

Skype for Business Online ports Client port requirements Required from client to Skype for Business Online Workload dependent Source port range per workload Workload Source IP Destination IP Source Port Destination port Audio Client IP O365 IPs 50,000-50019 TCP/UDP TCP 443, UDP 3478, 3479, 3480, & 3481, TCP/UDP 50,000-59,999 Video 50,020-50039 TCP/UDP Desktop Sharing/File Transfer 50,040-50059 TCP/UDP UDP 3479, 3480 & 3481

Hybrid Combination of all requirements Clients homed in service, need to connect to service AV Edge Server on premises needs required ports open Understand the troubleshooting scenario Where are the specific users or services located for a call that does not work Isolate the problem by trying different scenarios

Do’s and Don’ts for Service Direct connectivity required Clients need to directly connect to O365 Configure your firewalls, proxies, packet shapers etc. accordingly UDP and TCP Media will prefer (mostly) UDP TCP required for some scenarios and workflows Documented IPs and FQDNs “Office 365 URLs and IP address ranges” Subscribe to the RSS feed!

50k ports Port range open Port range closed 443 TCP 3478 UDP

Server: Edge Pool with DNS LB and NAT Firewall MUST allow hairpin: public IP to public IP External user might be behind firewall outside your control Outer Firewall 443 TCP 3478 UDP 50,000 port range 443 TCP 3478 UDP 50,000 port range Inner Firewall

Troubleshoot? Issue Server Service Inbound provisioning without “MRAS” AV Edge Server is not configured at pool This should not ever happen. Call support! Now! “MRAS” credentials not provided No connectivity between Front End and Edge internal interface No STUN/TURN candidates Clients unable to connect to server/service on UDP 3478/TCP 443 Packets being corrupted TURN candidates internal NATed IP address AV Edge Server not aware of external IP address

Where are the logs? Turn on logging first! Skype for Business 2016 %localappdata%\Microsoft\Office\16.0\Lync\Tracing Lync 2013/Skype for Business 2013 %localappdata%\Microsoft\Office\15.0\Lync\Tracing Lync 2010 (and earlier) “%userprofile%\tracing” Skype for Business for Mac Click “Collect Logs” in preferences

UccApilog.log search tips MRAS Finds inband provisioning MRAS request MRAS provisioning a=candidate Finds candidate exchange a=remote-candidate Finds promoted candidates that were used for call

More tools Synthetic transaction: Test-CsAVEdgeConnectivity http://technet.microsoft.com/en-us/library/jj205138.aspx Pre-Call Diagnostics http://technet.microsoft.com/en-us/library/dn451255.aspx Server: Telnet telnet <AV Edge internal FQDN> 5062 from Front End telnet <AV Edge internal FQDN> 443 from internal client telnet <AV Edge external FQDN> 443 from external client Service: Telnet telnet <AV Edge external FQDN> 443 from client

Resources Office Protocols http://msdn.microsoft.com/en-us/library/cc307432(v=office.12).aspx Skype for Business Debugging Tool (includes snooper) https://www.microsoft.com/en-us/download/details.aspx?id=47263 Office 365 URLs and IP address ranges https://support.office.com/en-us/article/Office-365-URLs-and-IP-address-ranges-8548a211-3fe7-47cb-abb1-355ea5aa88a2 Public recording of this presentation http://aka.ms/AVEdge

Related sessions Code Title Speaker BRK4011 Deploy ExpressRoute for Skype in Microsoft Office 365 Korneel Bullens BRK3054 Plan for Skype for Business cloud connectivity with Microsoft Office 365 Nikolay Muravlyannikov BRK3061 Ready your network for Skype for Business Online Hao Yan BRK2077 Get to know the Skype Operations Framework Ali Rohani BRK3058 Dig into the Skype Operations Framework Bryan Nyce

Session Objectives And Takeaways Tech Ready 15 2/25/2018 Session Objectives And Takeaways What is A/V Edge Server actually doing? How do we find the optimal media path? How do I read client logs? It’s interesting! Understand call flows It will help you troubleshoot! © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Deploy, ramp-up on new services and onboard new users with Microsoft FastTrack: http://fasttrack.microsoft.com/

Join the Microsoft Tech Community to collaborate, share, and learn from the experts: http://techcommunity.microsoft.com

Join the Skype for Business Community Microsoft Ignite 2016 2/25/2018 11:33 AM Join the Skype for Business Community Discover rich discussions and information sharing across customers, partners, and users IT Pro Forums User Forums Broadcast Updates Tips and Tricks Product Updates http://aka.ms/sfbcommunity © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Please evaluate this session 2/25/2018 11:33 AM Please evaluate this session Your feedback is important to us! From your PC or Tablet visit MyIgnite at http://myignite.microsoft.com From your phone download and use the Ignite Mobile App by scanning the QR code above or visiting https://aka.ms/ignite.mobileapp © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2/25/2018 11:33 AM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.