1. Traceable Measurements of 802.11 PHY Layer Performance
November 2004
doc.: IEEE t
Traceable Measurements of PHY Layer Performance
Dr. Michael D. Foegelle
ETS-Lindgren
Submission
Foegelle, ETS-Lindgren

2. Overview Introduction Real World Case #1: LOS Performance vs. Distance
November 2004
Overview
Introduction
Real World Case #1: LOS Performance vs. Distance
Traffic Generation
Measuring (Conducted) Transmit Power
Measuring (Conducted) Receiver Sensitivity
Other Considerations
Over-the-Air Testing
Summary & Conclusion
Foegelle, ETS-Lindgren

3. November 2004
Introduction
In order to offer test repeatability, and ensure that different labs can obtain the same test results, our test methodology must introduce traceability.
Ideally, test methodology should be developed to qualify the behavior of individual system components (AP, Client, etc.) using standardized methodology and traceable (i.e. calibrated) test instrumentation.
System performance can then be determined by combining metrics from individual components.
Foegelle, ETS-Lindgren

4. November 2004
Introduction
Requires writing standardized test methodology including methods of qualifying available test instrumentation used to create test traffic signals.
This presentation identifies system configurations and methodologies necessary to independently qualify the basic RF performance of the PHY layer of an individual DUT.
From there, the system level effect on higher layer metrics can be determined.
This methodology can then be expanded to encompass other PHY level effects (i.e. multipath)
Foegelle, ETS-Lindgren

5. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
Simple communication problem:
“Can you hear me now?”
Foegelle, ETS-Lindgren

6. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
Each leg of bi-directional communication link subject to
Available transmit power of source
Sensitivity of receiver
Gain of each device in LOS direction
Separation distance
Foegelle, ETS-Lindgren

7. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
In terms of the Friis Transmission Equation:
where:
Pr is the power at the output of RX antenna
Pt is the power at the input of TX antenna
Gt is the gain of TX antenna (in LOS direction)
Gr is the gain of RX antenna (in LOS direction)
r is the separation distance between antennas
 is the wavelength of the RF signal
Foegelle, ETS-Lindgren

8. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
The ratio of transmit to receive power is referred to as “path loss” and is given by:
This indicates that the effect of distance on the RF signal can be simulated easily by attenuating the signal between two devices.
Foegelle, ETS-Lindgren

9. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
Notes:
Electric field drops off as 1/r, but power is proportional to field squared and drops off as 1/r2. (4r/)2 represents the spatial drop-off in energy density due to the expanding spherical propagation.
Attenuation can also include effects of antennas. This points to performing cabled tests for most performance tests and only determining antenna performance for a subset of the possible metrics.
Foegelle, ETS-Lindgren

10. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
One simple method for performing a path loss performance test is to qualify complimentary DUTs (i.e. AP & Client) simultaneously.
Foegelle, ETS-Lindgren

11. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
There are a number of problems with this approach:
DUTs must be qualified as pairs.
5 APs and 5 Clients = 25 tests (excluding Ad Hoc, etc.) vs. only 10 tests if devices are qualified individually.
No way to separate out compatibility issues from performance issues.
No traceability to allow comparison with other test labs or confidence in results over time.
“Golden Unit” as reference may give repeatability, but unless shared with other labs, no way to compare results.
Damage to “Golden Unit” loses point of reference.
Foegelle, ETS-Lindgren

12. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
Problems with DUT pair approach (continued):
Difficult to know if samples are representative of production.
Variation in both samples are combined in test results (Hi-Hi, Hi-Lo, Lo-Hi, Lo-Lo)
Higher overall measurement uncertainty.
Limited range of tests parameters that can be varied.
Manufacturers unlikely to publicize test results of their product communicating with competition’s product.
Foegelle, ETS-Lindgren

13. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
Solution is to qualify each DUT separately and in such a way that RF performance of pair can be determined.
Foegelle, ETS-Lindgren

14. Real-World Case #1: LOS Performance vs. Distance
November 2004
Real-World Case #1: LOS Performance vs. Distance
A traceable way must be found to:
Determine available transmit power of DUT
Determine sensitivity of DUT
Determine antenna related TX/RX performance
Determine metrics directly related to each.
From there, we can apply Friis transmission equation to determine paired metrics as function of distance.
Foegelle, ETS-Lindgren

15. November 2004
Traffic Generation
All of these steps are going to require a method of traffic generation and monitoring. Traffic generator (TG) must have:
Fully compliant protocol operation.
Capability of generating/monitoring traffic to the maximum level expected for DUT.
Stable radio TX power.
Sufficient dynamic range/sensitivity for expected RX signals.
Foegelle, ETS-Lindgren

16. November 2004
Traffic Generation
Basic traffic generator could be PC and suitable AP or client with suitable traffic generation software.
Foegelle, ETS-Lindgren

17. November 2004
Traffic Generation
As will become apparent, a more advanced traffic generator may be necessary to meet all requirements of traceable single DUT measurements. Important features include:
Separate transmit and receive ports for radio.
Added stability of radio over that found in commercial products.
Lower level control of traffic.
Traffic dependent triggering signals.
There are already products on the market that are likely to meet these requirements.
Foegelle, ETS-Lindgren

18. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Below is one possible system for measuring DUT power using off-the-shelf components.
Foegelle, ETS-Lindgren

19. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Measure cable/coupler path loss and apply.
Calibrated Receiver provides traceability.
Allows comparison of lab results with confidence.
Measurements can be expressed within a given uncertainty level.
Number of possible choices for receiver:.
Spectrum Analyzer
Vector Signal Analyzer
Power Meter
Foegelle, ETS-Lindgren

20. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Spectrum Analyzer
Traditional instrument for frequency dependent measurements.
Readily available.
Newer models have low uncertainties (<0.5 dB compared to older models ~2 dB).
Front end bandwidth narrower than channel (3 MHz RBW vs. 5 MHz channel).
Requires integrated channel power measurement.
Synchronization may be an issue.
Foegelle, ETS-Lindgren

21. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Vector Signal Analyzer
Modern enhancement over spectrum analyzer.
Provides vector measurement for phase modulation analysis.
Foegelle, ETS-Lindgren

22. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Power Meter
Usually considered most accurate method of measuring total power.
No frequency discrimination.
Measures out-of-band signals too.
Variety of sensors and designs.
Wide variety of responses to different input signal types.
Newer units designed with options for measuring certain signal types.
Foegelle, ETS-Lindgren

23. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Power Measurement Issues
uses bi-directional traffic in same channel
Need to be able to discriminate between forward (TG TX) and reverse (DUT TX) communication.
Directional coupler provides some signal level discrimination for conducted tests (problem for OTA).
Ideally, TG would generate synchronization signal to trigger power measurement when packet RX starts.
max packet length is short.
Channel integration requires time to sweep across channel. Need continuous transmission.
Foegelle, ETS-Lindgren

24. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Power Measurement Issues
FHSS measurement difficult/long
Need to measure power at each hop frequency to have legitimate measurement of total channel power.
Should measure power at each traffic level.
Different modulations could easily have different power levels.
Need a way to control which traffic level the DUT will transmit at during power measurement.
Foegelle, ETS-Lindgren

25. Measuring (Conducted) Transmit Power
November 2004
Measuring (Conducted) Transmit Power
Power Measurement Issues
Need a traffic generator that can synchronize measurement to DUT transmission.
Foegelle, ETS-Lindgren

26. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Need to selectively control forward power level of TG over RX dynamic range of DUT.
Attenuate TX path of TG w/o attenuating RX path.
Ideally, TG radio would have separate transmit and receive ports, allowing insertion of attenuator on one leg.
Foegelle, ETS-Lindgren

27. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Is possible to develop bi-directional amplifier.
Common design uses sensors for direction control.
May be possible to control from TG.
Foegelle, ETS-Lindgren

28. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Traceability is added to test system by calibrating TG/system TX power using suitable receiver.
Ideally TG can generate forward traffic for direct measurement.
Measured for every data rate of TG.
Foegelle, ETS-Lindgren

29. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Traceability (continued).
Measuring through amp, attenuator (at all levels), & cables creates transfer standard from receiver absolute accuracy and linearity.
Assumes stability of TG output and stability of any amplifier added to system.
Foegelle, ETS-Lindgren

30. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Sensitivity Metric Definition
Traditionally, sensitivity measurement would entail determining what received level produces a given bit or frame error rate.
In this case, this would imply special PHY level functionality in DUT (Test API).
Must determine sensitivity of each data rate.
In lieu of BER/FER test, throughput test offers higher level metric capable of determining similar information.
Slow by comparison.
Foegelle, ETS-Lindgren

31. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Throughput as Sensitivity Metric
Only throughput to DUT matters.
TG must not introduce additional delay that could taint results.
Ideally, TG would be able to be locked to a given data rate and test throughput at that data rate.
Eliminates “mixing” of data rates.
Allows “pure” measurement of each data rate sensitivity independent of any transition logic.
Avoids transition delays and data rate oscillation.
Foegelle, ETS-Lindgren

32. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Throughput as Sensitivity Metric (Cont’d)
Packet size could have significant effect on throughput and throughput measurement.
Smaller packet = more overhead but better error rate resolution
Larger packet = higher peak throughput but coarser resolution of error rate. As BER increases, larger packets may never get through at all (function of packet length vs. MTBE). Results in slower measurement and/or poorer statistics.
Foegelle, ETS-Lindgren

33. Measuring (Conducted) Receiver Sensitivity
November 2004
Measuring (Conducted) Receiver Sensitivity
Complete Test System & Procedure
For each attenuation (DUT RX power level)
Measure throughput to determine approximate error rate
May want range of throughput vs. attenuation or target specific throughput (error rate).
Foegelle, ETS-Lindgren

34. Other Considerations (Part 1)
November 2004
Other Considerations (Part 1)
Bridging the Gap
We intentionally removed data rate transition logic from sensitivity measurement.
Transition decision occurs on sender (TG in this case).
Need to test transition logic of DUT to marry with sensitivity information of a paired DUT (measured separately) in order to fully predict behavior of pair.
(Not sure how we do this at this time.)
Combine TX power and RX sensitivity of pair to determine limiting performance of pair
Foegelle, ETS-Lindgren

35. Other Considerations (Part 1)
November 2004
Other Considerations (Part 1)
Bridging the Gap
Combine TX power of one device with RX sensitivity of second device (and vice-versa) through Friis transmission equation to determine limiting performance of pair in each direction.
Requires knowledge of LOS antenna gain for each.
Separate measurement OK for estimates.
Direct performance measurement important (see 11-04/675r1)
See next section for methodology differences.
Foegelle, ETS-Lindgren

36. Other Considerations (Part 1)
November 2004
Other Considerations (Part 1)
Other Metrics
While the sensitivity measurement was specifically looking for a way to duplicate the results of a BER measurement with existing technology, there may be other low level metrics that vary differently from throughput at sensitivity.
In general, this test system should allow evaluation of the RF behavior of any data metric.
Expansions can be made to this system for evaluation of other environmental effects such as NLOS, etc.
Foegelle, ETS-Lindgren

37. November 2004
Over-the-Air Testing
See other presentations for detailed considerations of OTA testing.
This section will primarily show modifications to the test systems shown previously that are necessary in order to perform OTA testing.
OTA testing is a confirmation of total DUT performance and would not normally be used for more than the two metrics listed here.
Exception may be DUTs with integral antennas and no separate RF port.
Foegelle, ETS-Lindgren

38. Over-the-Air Testing Radiated Power Measurements November 2004
Foegelle, ETS-Lindgren

39. Over-the-Air Testing Radiated Sensitivity Measurements November 2004
Foegelle, ETS-Lindgren

40. November 2004
Summary & Conclusion
This presentation shows basic methodology for obtaining traceable measurement results for RF performance of individual devices.
This methodology provides necessary input to system performance prediction algorithms.
Currently available test instrumentation can be used to do this testing, although there are limitations. This presentation points the way to enhancements needed in available equipment.
Additional metrics may be determined in a manner similar to that shown here.
Foegelle, ETS-Lindgren