1. Metrics for Characterizing BSS Transition Time Performance
August 2004
doc.: IEEE /999r1
August 2004
Metrics for Characterizing BSS Transition Time Performance
Date: September 7, 2004
Authors:
Charles R. Wright, Azimuth Systems, Inc., Acton, MA
Chris Polanec, InterOperability Lab, Durham, NH
C. Wright, Azimuth Systems
C. Wright, Azimuth Systems

2. August 2004
doc.: IEEE /999r1
August 2004
Abstract
TGr is currently planning a selection process for candidate fast BSS transition protocols. This presentation proposes a metric and test methodology for characterizing the performance of real equipment using such a transition protocol. It is based on repeated measurements of a refined definition of BSS transition time as defined in 11-04/805r2. It produces a statistical characterization useful to system designers as well as for comparison of candidate fast BSS transition protocols.
C. Wright, Azimuth Systems
C. Wright, Azimuth Systems

3. Outline Background and definitions The metrics Test configuration
August 2004
Outline
Background and definitions
The metrics
Test configuration
Other relevant issues
Conclusions
C. Wright, Azimuth Systems

4. August 2004
Background
Held three ad hoc telecons between Portland and Berlin meetings
Discussed various issues with measuring transition time
This presentation represents a synthesis of the outcomes from these discussions on the issues
Thanks to the other ad hoc attendees:
Mike Montemurro, Fanny Mlinarsky , Chris Polanec, Jeremy Spilman, Clint Chaplin
C. Wright, Azimuth Systems

5. Fast BSS Transition Time Definition
August 2004
Fast BSS Transition Time Definition
From 11-04/805r2, 1.5.4, we have “Fast-BSS-Transition-Time”:
Security not enabled:
the period between the last possible point in time (tA) where STA-TE communication can pass through the Old-AP, to the first possible point in time (tB) where STA-TE communication can pass through the New-AP
Security enabled:
the period between the last possible point in time (tA) where STA-TE communication can pass through the Old-AP, to the point in time (tB) where the first MSDU can pass through the controlled port
TE = “Traffic Endpoint”
C. Wright, Azimuth Systems

6. August 2004
As defined, Fast BSS Transition Time is useful for analysis, but is not measurable
The “last possible point in time where STA-TE communication can pass through the old AP” is not externally observable
Also applies to “first possible point in time where STA-TE communication can pass through the new AP”
What we can observe is the last MSDU transmitted over the air with apparent success
Last possible instant where frames can pass through old AP
First possible instant where frames can pass through new AP
“Fast BSS Transition Time”
. . .
. . .
MSDU
ACK
MSDU
ACK
Old AP
New AP
C. Wright, Azimuth Systems

7. We propose new transition time definitions
August 2004
We propose new transition time definitions
“Fast BSS Transition Time” as defined in 11-04/805r2, is really the “Theoretical BSS Transition Time”
What is measured using the last successful MSDU on the first channel and the first successful MSDU on the new channel is: “Observed BSS Transition Time”
Last possible instant where frames can pass through old AP
First possible instant where frames can pass through new AP
Theoretical BSS Transition Time
. . .
. . .
MSDU
ACK
MSDU
ACK
Observed BSS Transition Time
C. Wright, Azimuth Systems

8. BSS Transition Time Metrics
August 2004
BSS Transition Time Metrics
Two forms of this metric, depending on the capabilities of the client
Minimum BSS Transition Time
Is a single value
Can be measured if client can generate arbitrary traffic rates
Observed BSS Transition Time Distribution
Produces a cumulative distribution for a specified frame rate
Can be measured with any client
C. Wright, Azimuth Systems

9. Measuring Minimum BSS Transition Time
August 2004
Measuring Minimum BSS Transition Time
Send periodic traffic and measure the transition time
Traffic interval should be greater than the expected transition time
Measure transition time “many times”; note minimum value
Decrease the frame interval and do this again
Repeat until observed transition time no longer decreases with decreased frame interval
Minimum value of transition time for this frame interval is estimate of Theoretical BSS Transition Time
Frame Interval
. . .
Transition Period
. . .
MSDU
ACK
MSDU
ACK
Old AP
New AP
C. Wright, Azimuth Systems

10. Fixed frame rate traffic is very common
August 2004
Fixed frame rate traffic is very common
Wireless voice handsets are likely the “killer app” for TGr, at least right now
Voice traffic is periodic and fairly low rate (10 to 30 ms frame interval)
Voice handsets have no real need to vary frame rate
Minimum transition time might vary with traffic load
Under high loads, may be difficult for client to scan other channels
Need a metric that is meaningful for fixed frame rate traffic
Means metric can be targeted for a known and specific application
C. Wright, Azimuth Systems

11. Observed BSS Transition Time metric is inherently statistical
August 2004
Observed BSS Transition Time metric is inherently statistical
Transition may involve processes that take variable time
STA/AP may introduce jitter in the target frame interval
STA may buffer frames during transition and not emit traffic at a periodic interval
STA may decide to roam at any time instant between transmitted frames
Observed BSS Transition Time
. . .
Old AP
Transition Period
New AP
Observed BSS Transition Time
. . .
Old AP
Transition Period
New AP
Need to measure transition time repeatedly to characterize performance
C. Wright, Azimuth Systems

12. Observed BSS Transition Time Distribution
August 2004
Observed BSS Transition Time Distribution
Choose or record the frame rate (frame interval)
Measure and record transition time for many trials
Display results as a cumulative distribution
Read result as (e.g.)
“64% of the time, the transition time for Proposal 2 is less than 40 ms”, or
“36% of the time, the transition time for Proposal 2 is more than 40 ms”
Frame Interval: 20 ms
*** Completely fictitious data! ***
C. Wright, Azimuth Systems

13. Why a Cumulative Distribution of the Transition Time?
August 2004
Why a Cumulative Distribution of the Transition Time?
Considered counting packets lost during transition, but
Requires intimate knowledge of the protocol to count lost packets
On uplinks, STAs might decide to buffer traffic queued during a transition, so frame(s) might not be lost, only delayed
What really matters depends on the application, anyway
Some codecs can cover frame loss better than others
Some devices may be smart about buffering during transitions
CDF of Observed BSS Transition Time allows system designers to use this measure of network interruption time and full knowledge of their own devices to calculate outage rates
C. Wright, Azimuth Systems

14. Cumulative distribution is useful for system engineering
August 2004
Cumulative distribution is useful for system engineering
For example, assume G.711 (20 ms) traffic and a 1 packet-deep jitter buffer
If transition takes longer than time it takes to receive 2 packets, then a codec underrun occurs
Cumulative distribution can answer these questions:
What is the probability that transition causes the jitter buffer to empty and we underrun for one frame?
What is the probability we underrun for 2 frames? Etc.
C. Wright, Azimuth Systems

15. Example from Current Technology
August 2004
Example from Current Technology
40 Transitions
Mix of 11a/b/g clients with various APs
C. Wright, Azimuth Systems

16. August 2004
Test Configuration
C. Wright, Azimuth Systems

17. Ethernet Sniffer; Traffic Endpoint
August 2004
Test Configuration
Ethernet Switch
AP1
AP2
Ethernet Sniffer; Traffic Endpoint
Sniffer
Sniffer
Atten1a
Atten2a
Combiner
Combiner
Atten1b
Atten2b
Placement of wireless sniffers in the RF network allows reception of AP and client no matter what the attenuator settings are
Combiner
Test head with roaming client STA
C. Wright, Azimuth Systems

18. August 2004
Some Requirements
Design test so that is possible using passive observation (airlink and ethernet)
Test run with under optimal channel conditions and no background load
Sniffers must be time synchronized
Various ways to do this
No more than one Ethernet switch on the wired backbone to minimize any switching delays
C. Wright, Azimuth Systems

19. Attenuator Sweep Profile
August 2004
Attenuator Sweep Profile
Attenuators start off such that client associates with AP1
Atten1a = Atten1b = minimum; Atten2a = Atten2b = maximum;
After dwell interval, attenuators sweep to make AP2 more favorable for client
Atten1a, Atten1b both increase at same rate; Atten2a, Atten2b both decrease at same rate
Eventually, client transitions to AP2
After attenuators reach max/min and another dwell interval, process reverses to “push” client back to AP1
Client Association:
AP1
AP2
max
min
Atten
Attenuator Sweep Time
Dwell Interval
Atten2a+Atten2b
Atten1a+Atten1b
C. Wright, Azimuth Systems

20. Attenuator Considerations
August 2004
Attenuator Considerations
Total of four programmable attenuators are recommended
Idea is to make sure airlink sniffer can always hear both the roaming client and the AP
Can probably get by with only two…
Replace Atten1a and Atten2a with a fixed attenuator, or Atten1b and Atten2b
… but using fixed attenuators can make it more difficult to cause the roaming client to go completely out of range of the AP
Designer of test system has to “add up the dBs” to make sure it will work out!
C. Wright, Azimuth Systems

21. Other Issues Uplink, downlink or bidirectional test traffic?
August 2004
Other Issues
Uplink, downlink or bidirectional test traffic?
Speed of attenuator variation?
Power save
Issues with QoS and Security features
Architecture of network of APs
C. Wright, Azimuth Systems

22. Uplink, Downlink or Bidirectional?
August 2004
Uplink, Downlink or Bidirectional?
Should transition time metric be measured separately for each of the above traffic orientations?
Because of wireless voice handsets, bidirectional performance is most interesting
Observed BSS Transition time should be measured simultaneously for up and downlink frames
Results are reported separately for up and down link
. . .
. . . Up Up Up Up
. . .
. . .
Down
Down
Down
Down
. . .
. . . Up
Down Up
Down Up
Down Up
Down
C. Wright, Azimuth Systems

23. Speed of Attenuator Variation
August 2004
Speed of Attenuator Variation
Want to measure only the BSS transition time
Do not want measurement influenced by scanning, pre-auth, neighborhood lists, etc.
These factors are out of scope for TGr
Decrease in signal level only serves to stimulate BSS transition
There may be other reasons to transition, but again, we’re interested only in the observed BSS transition time
Leads to question for test: precisely how long to dwell with one AP before stimulating the transition?
With current equipment, about 10 sec seems to work, assuming also the attenuators change at ½ dB per second
C. Wright, Azimuth Systems

24. What about Power Save modes?
August 2004
What about Power Save modes?
Power save mechanisms
Legacy
802.11e Scheduled APSD
802.11e Triggered APSD
In the ad hoc telecons, we could determine no issues where power save would meaningfully affect the transition time metric
Fast transition matters for stations sending time-critical traffic
Immediately roaming after coming out of dormancy does not seem to be time critical
C. Wright, Azimuth Systems

25. August 2004
Architecture of APs
Thin AP architectures may interfere with verification of network connectivity
C. Wright, Azimuth Systems

26. Issues with QoS and Security Features
August 2004
Issues with QoS and Security Features
802.11e NoAck policy will cause problems with knowing whether there is actual network connectivity
Uplink traffic work-around: observe data frames on the wired network
Downlink traffic work-around: none – we can only assume the STA received the frame if it was transmitted by the AP
We don’t know how popular the NoAck policy will become
For i, we must assume that after a transition, the PAE on a receiving device will open a controlled port for data frames
Frames are ACK’d but may not pass through if the PAE blocks them
Uplink traffic work-around: must confirm successful transmission by observing Ethernet frames
Downlink traffic work-around: none – same issue as with NoAck
We expect security will be very popular
C. Wright, Azimuth Systems

27. Conclusions Defined the test setup and methodology
August 2004
Conclusions
Defined the test setup and methodology
Defined two new terms for BSS transition times
Theoretical BSS Transition Time
Observed BSS Transition Time
Defined two new metrics
Minimum BSS Transition Time (estimate of Theoretical)
Observed BSS Transition Time Distribution
Other airlink events can be measured with the test setup, but are not part of the metrics
tretry, tscan, tassociate, tdata (11-04/748r1)
C. Wright, Azimuth Systems

28. References 11-04/0086r3: Measurement of 802.11 Roaming Intervals
August 2004
References
11-04/0086r3: Measurement of Roaming Intervals
11-04/0748r1: Test Methodology for Measuring BSS Transition Time
11-04/xxxr0: BSS Transition Time Test Methodology
11-04/805r2: TGr Minimum Requirements
RFC 2285: Benchmarking Terminology for LAN Switching Devices
C. Wright, Azimuth Systems