1. WiFi Troubleshooting & Performance Monitoring
Case Study : Troubleshooting wireless issues via Mojo Aware
- Asvin

2. Case studies
In this session, we will cover how Mojo Aware helped system administrators to identify following issues in corporate environment:
Scenerio1: Wireless connection is lost when the wireless session times out in Windows 7 or Windows Server 2008 R2
Scenerio2: Issues with clients staying connected to an Access Point that has bad signal (Sticky Clients)

3. Case Study: Window 7 Clients loses WiFi connectivity
Vertical: SME branch office
Problem
Wireless connection is lost when the wireless session times out in Windows 7 or Windows Server 2008 R2
Flag
Mojo Aware flagged Invalid MIC failure on dashboard
Cause
Win 7 clients randomly introduce invalid MIC causing Mojo AP’s to reject M2 messages

4. Learning Objectives User impact of random client disconnection
Problem Statement
Detailed description of EAPOL 4-Way handshake functionality
Symptoms
Cause
How Mojo Aware quickly identifies Invalid MIC failures and promptly alerts administrator

5. User Impact of the Problem
Few minutes disruption of WiFi during office hours
Issue occurs randomly on Windows 7 laptop
Takes nearly 1 to 2 mins for the laptop to reconnect to wireless network

6. Debugging “Invalid MIC failure”
Problem Statement
Few minutes random disruption of WiFi
Windows 7 clients connecting to wireless networks configured with WPA2 and session timeout may get disconnected during the key exchange after re- authentication
Over the air, M1 and M2 packet retried several times causing delay in reconnection
AP logs during problem:

7. EAPOL 4-way Message in detail (M1)
Apply display filter EAPOL-Key messages using “eapol.keydes.type == 2” wireshark display filter.
Message 1 (M1)
Authenticator sends EAPOL-Key frame containing an ANonce(Authenticator nonce) to supplicant.
With this information, supplicant have all  necessary input to generate PTK using pseudo-random function(PRF)

8. EAPOL 4-way Message in detail (M2)
Message 2 (M2)
Supplicant sends an EAPOL- Key frame containing SNonce to the Authenticator.
Now authenticator has all the inputs to create PTK.
Supplicant also sent RSN IE capabilities to Authenticator & MIC
Authenticator derive PTK & validate the MIC as well.

9. EAPOL 4-way Message in detail (M3)
Message 3 (M3)
If necessary, Authenticator will derive GTK from GMK.
Authenticator sends EAPOL- Key frame containing ANonce, RSN-IE & a MIC.
GTK will be delivered (encrypted with PTK) to supplicant.
Message to supplicant to install temporal keys.

10. EAPOL 4-way Message in detail (M4)
Message 4 (M4)
Supplicant sends final EAPOL-Key frame to authenticator to confirm temporal keys have been installed.
From this point onwards data frame will be encrypted using PTK or GTK (depending upon unicast or multicast/broadcast frame)

11. Symptoms
This issue occurs when a Windows 7-based computer is connected to a wireless network by using the WiFi- WPA2 protocol and the wireless access point (AP) starts a new exchange of WPA2 keys. In the four-way handshake, the Windows 7-based computer sends a Message 2 (M2) with an invalid message integrity check (MIC) Note This issue may occur every 12 hours or more frequently, and it takes one minute to regain the network connectivity. 

12. Cause
This issue occurs because the WPA2 key context is not set correctly before the four-way handshake rekeys. Certain variables are not reset after the previous four-way handshake. This causes the secure bit to be set incorrectly and the stale Pairwise Transient Key (PTK) to be used to calculate the MIC in the M2 key messages. APs reject the M2 messages because of these errors.

13. Root cause analysis
Microsoft confirmed this bug and fixed in Hotfix Win7 release.
Win7 laptop required above hotfix upgrade.

14. How Mojo-aware quickly identifies Invalid MIC failures and promptly alerts administrator
Mojo Aware pinpoint exact cause of failure and saves administrator time and effort for debugging
Wireless capture is saved here!

15. Aware: Capture for corresponding failure
Mojo Aware display exact packet capture during problem statement without applying any display filters!

16. Mojo Aware advantages
Promotes actual cause of issue, not just client connectivity failure
Administrator identifies complex EAPOL issues in single glance
Save time and debugging effort without using real time wireless sniffer
Wireless client failure logs preserved in cloud and can be looked and accessed any time.

17. Case Study2: Bad signal(Sticky clients) causing network slowness
Vertical: SME branch office
Problem
Issues with clients staying connected to an Access Point that has bad signal (Sticky Clients)
Flag
Mojo Aware flagged sticky clients on dashboard
Cause
Clients still have good signal strength to far away AP so do not disassociate

18. Learning Objectives User impact of bad signal and sticky client
Problem statement
What is sticky client
Symptoms
Cause
Resolution
How Mojo-Aware quickly identifies sticky clients and promptly alerts administrators

19. User Impact of the Problem
Far away clients contend with rising error rates due to the lower signals
Overall wireless efficiency of the cell is reduced as clients wait longer than they should for a slower speed client to send its data
Key to high performance WiFi network is airtime efficiency
Even a small number of sticky clients, using sub- optimal speeds, can very quickly drag down the performance of Wi-Fi network

20. Debugging “Bad signal and network slowness”
Problem Statement
Customer reported “Overall network slowness ”
Basic file copy operation within the network takes long time to transfer.

21. Sticky clients and roaming decision
Wireless clients tend to hang on to the original access point they associated with, rather than moving to a nearby AP that would generally be a better choice for them.
Roaming Decision is a client decision, not a network decision
AP don’t tell client when to roam – the network has to respect the wishes and behavior of client devices.

22. Symptoms Identifying Sticky clients - behavior
Do not probe on other channels
Probe infrequently
Remain associated to an AP even through better Aps are available
Transmit on low PHY rate consuming more air

23. Sticky client end user impact

24. Sniff capture with sticky client in network
Low Data rate
Low RSSI

25. Mojo Aware
Mojo Aware quickly identifies sticky clients and promptly alerts administrators to take appropriate action.

26. Resolution Enable Smart Steering Enable Min Association RSSI
Disassociate “Sticky clients”
Prevent them from re-associating to the AP
Encourage/Force roam to better AP
Configured per SSID (Enable/Disable)
Enable Min Association RSSI
RSSI Threshold
Reduce the number of probe/assoc response
Prevent clients with RSSI below the threshold from associating

27. Sniff Capture-After Enabling smart steering
Deauth after enabling smart steering

28. Mojo Aware - After Enabling Smart steering

29. Mojo Aware Advantages
Warns the administrator about network slowness (eg. Sticky client)
Ability to quickly identify the WLAN issues
Ability to quickly detect total no of sticky or bad clients in the network
Ability to isolate problematic clients with complete details include packet captures and system logs for further analysis.

30. Thank You

31. Backup slides

32. EAPOL 4-Way Handshake functionality
Process:
The AP sends a nonce-value to the STA (ANonce). The client now has all the attributes to construct the PTK.
The STA sends its own nonce-value (SNonce) to the AP together with a MIC, including authentication, which is really a Message Authentication and Integrity Code: (MAIC).
The AP sends the GTK and a sequence number together with another MIC. This sequence number will be used in the next multicast or broadcast frame, so that the receiving STA can perform basic replay detection.
The STA sends a confirmation to the AP.

33. What is session timeout?
Session timeout means that authenticated user session expires in 1800 seconds based on implementation, it is not an activity or idle timeout.  So depending on authentication method, this could cause client to disconnect.

34. Root cause analysis
Microsoft confirmed this bug and fixed in Hotfix Win7 release.
Win7 laptop required above hotfix upgrade.
Per Microsoft, this issue can also be mitigated by reducing the EAPOL key retransmission timeout. The issue was first seen with timeout value of 3ms. When reducing this value to 1msec the issue was fixed.
Note:
Do be aware that reducing this value might negatively impact key negotiations with some very old and slow clients.

35. How Mojo-aware quickly identifies Invalid MIC failures and promptly alerts administrator
Mojo-aware pinpoint exact cause of failure and saves administrator time and effort for debugging