1. 31-Jan-2014 Fanny Mlinarsky octoScope, Inc.
Testing MIMO Radios
Day 5: Benchmark Testing of MIMO Performance
31-Jan-2014 Fanny Mlinarsky
octoScope, Inc.

2. Wireless Test Challenges
Why is getting repeatable and consistent measurements is next to impossible in open air conditions?
Modern wireless devices are designed to automatically adapt to the changing channel conditions.
Adaptation algorithms programmed into the baseband layer of these radios are complex and sometimes get into unintended states.
Wireless environment is time-, frequency- and position- variable in terms of path loss, multipath, Doppler effects and interference, often stumping the decision logic of the adaptation algorithms.

3. Benchmark Testing
Goals of benchmark testing is to objectively compare performance of devices from different vendors
Under best conditions
Under challenging conditions

4. Ideal Benchmarking Testbed
Ideal benchmarking testbed must:
Create conditions under which throughput of the DUT is maximized
Emulate a range of realistic conditions, including path loss, multipath, noise and interference
Guarantee repeatable measurements reproducible at labs around the world

5. Evolution of Wireless Test Methods
MIMO OTA
Multi Path Emulator (MPE)
SISO conducted
Isolation box
New generation wireless testbeds must support MIMO OTA testing to accommodate MIMO and multi-radio devices with internal antennas.

6. Factors Impacting MIMO Throughput
Explanation/Impact
Notes
MIMO channel correlation
Function of several variables including device antenna spacing, antenna polarization and multipath
The lower the correlation the higher the throughput
Angular spread of the received signal
Related to correlation and strongly influenced by multipath in the channel
Multipath causes signal to bounce around and arrive at different angles, thereby widening the angular spread at a receiver. Typically, the wider the angular spread the higher the MIMO throughput.
Device antenna spacing and orientation
Related to angular spread and correlation
MIMO throughput will vary vs. device orientation and antenna spacing. Typically, the wider the antenna spacing the lower the correlation and the higher the throughput.
Antenna polarization
Vertical, horizontal or circular
Cross-polarization (vertical and horizontal) is sometimes used to lower MIMO correlation, thus enabling spatial multiplexing. Multipath reflections can alter polarization.
Noise and interference
High noise power with respect to signal power results in low SNR (signal to noise ratio)
MIMO devices can adapt to the environment by selecting the most suitable mode of operation (e.g. TX diversity in low SNR conditions; spatial multiplexing in high SNR, low correlation conditions).
Motion of devices or multipath reflectors
Causes Doppler spread of the signal
OFDM signaling is sensitive to Doppler spread. Throughput should be measured in a variety of Doppler environments.
Delay spread of reflections
Causes clusters of reflections to arrive at the receiver at different times
Delay spread is higher for larger spaces (e.g. outdoors) than for smaller spaces (e.g. home environment)

7. Maximizing MIMO Throughput in a Small Anechoic Chamber
Factors
Corresponding chamber capabilities
MIMO channel correlation
Low correlation can be achieved via wide spacing of test antennas
Angular spread of the received signal
(1) Test antenna array is close to the DUT antennas, widening the angular spread of the LOS signal; (2) chamber geometry creates -20 dB reflections surrounding the DUT. Both of these factors maximize angular spread of the test channel thereby maximizing MIMO throughput.
Device antenna spacing and device orientation
Test antenna spacing can be adjusted. DUT can be rotated with respect to test antennas.
Antenna polarization
Test antennas can be cross-polarized. However, reflections inside the chamber and metal surfaces in the DUT (ground planes, batteries, etc.) will also alter polarization of the signal that reaches the DUT antennas.
Noise and interference
Can be injected in a controlled manner via extra RF ports.
Motion of devices or multipath reflectors
Can be implemented via phase dithering of the signal or by using a fader.
Delay spread of reflections
octoBox MPE module models IEEE standard delay spread.

8. Factor: Angular Spread
Shorter distance = wider angular spread = higher throughput 4” 8” 8’
Open air - throughput decreases with distance
Source of plot: SmallNetBuilder.com measurement (linksys_ea6500_5ghz_80mhz_up_mpe_vs_openair.png)

9. Wide Angular Spread in a Small Anechoic Chamber
DUT
Test antenna spacing can be controlled
octoBox chamber

10. Multipath in a Small Chamber
Realistic magnitude of reflections in a typical house.
Reflections surround the DUT from walls, ceiling and floor, creating a wide-angular- spread environment modeling non-line-of- sight (NLOS) multipath reflections to help achieve maximum throughput.
Line of sight (LOS) transmission between the test antennas and the DUT is a function of the proximity of test antennas to the DUT.
-20 dB absorption on frequencies 1-6 GHz
-15 dB absorption on frequencies down to 700 MHz
2.25” gradient absorptive foam covers inside surfaces
But what about delay spread of the multipath?

11. Multipath in a Small Chamber
octoBox
DUT AP
Test Antennas
Master Client
Traffic TX/RX
Remote Desktop
Attenuators
Ethernet Filter
Multipath segment
Testbed being used for benchmarking
MPE = multi path emulator

12. octoBox MPE Response vs. IEEE Model B

13. octoBox MPE Frequency Response

14. Beware of Adaptation Algorithms
Initiation algorithm 4” 8” 8’
Open air - throughput decreases with distance

15. Beware of Adaptation Algorithms
Example of throughput measurement of an ac link using IxChariotTM. In this example the test conditions are static, but it appears that the adaptation algorithm of the TX DUT keeps making adjustments resulting in throughput fluctuations vs. time.

16. Benchmarking of MIMO Throughput
Source:

17. SmallNetBuilder AP Testbed
Client: ASUS PCE-AC66; driver Win *
Test client machine has Win Defender and Firewall disabled; runs no antivirus; does not run Windows update. It is dedicated solely to being a test client.
Traffic generator/analyzer: IxChariot
Measured throughput strongly influenced by the IxChariot test file size.
SmallNetBuilder uses 2,000,000 to 5,000,000 Byte file size.
The high performance throughput script uses 10,000,000 Byte file size by default.
* Subject to change with new standards and technology releases

18. Isolating DUTs in a Wireless Testbed
Loss of testbed dynamic range due to crosstalk

19. How to Select RF Isolation Chamber
Two issues to be aware of:
Isolation specifications often don’t include the impact of data and power cables that must penetrate the walls of the chamber to power and control the DUT inside during the test.
OTA support requires high isolation, absorption and special conditions to enable high MIMO throughput.
Choose peel-and-stick gasketing that’s easy to replace
Plastic rails and hardware to avoid reflections
OTA = over the air

20. Wireless Performance Metrics
Metrics for wireless devices and systems include
Throughput
Packet error rate (PER)
Range
Roaming speed
Mesh operation
Application specific operation, for example DSRC automotive networks
Test should be performed with and without channel impairments
DSRC = direct short range communications

21. Wireless Testbed Building Blocks
Testbeds should be versatile – made up of building blocks suitable for configuring a variety of test topologies
A single large screen room only supports a single airlink

22. Roaming Test Configuration
RF attenuators
Controller or Ethernet switch
Isolation chambers
RF port
Spare combiner ports for connecting a sniffer
AP1
ATTEN1
Traffic source/sink
Ethernet
AP2
1:4 RF combiner
ATTEN2
Traffic source/sink
Client/bridge under test
Feed-through Ethernet filters
USB
octoBox Stackable

23. octoBox Roaming Test Script
Programmable attenuator ramp
Ping request-response delay
Connection outage due to roaming

24. Testbed Considerations
Extra RF ports are useful to inject interference or connect monitor probes
An extra filtered USB connections can house a USB module for traffic monitoring

25. Wireless Mesh Testing Wireless Mesh Metrics Self-healing, self-forming
Throughput, QoS vs. hops
Throughput, QoS vs. range
Routing efficiency
Dealing with interference

26. Wireless Mesh Test Configuration
Maximize attenuators to force auto-re-routing of traffic flow to test self-healing 1
RF splitters used to direct signal to multiple neighboring devices
Radios are paced in isolation chambers
Fixed attenuators to set traffic flow via one branch or another to test self configuration
octoBox quadStack isolation enclosures with built-in RF combiners and attenuators

27. Connected Car Testbed
Emulate groups of connected vehicles traveling over a busy highway interchange

28. Example Large-scale DSRC Testbed
Emulate motion of groups of cars with respect to other groups
8-port wireless channel emulator
Motion
Multipath
Noise
Up to 24 radio modules in each chamber
192 radios in the testbed

29. Summary Wireless technology is spreading into many applications
Different benchmarks will be needed to ensure proper operation of complex radios used in hostile airlink conditions
Now is a good time to think about a versatile wireless testbed that can accommodate needed tests and provide reproducible measurements in labs around the world.

30. For More Information
To download white papers, presentations, test reports and articles on wireless topics, please visit