Enabling Prediction of Performance September 2004 Enabling Prediction of Performance September 16, 2004 Roger Skidmore Wireless Valley Communications, Inc. Roger Skidmore

Abstract Introduction What is Performance “Prediction”? September 2004 doc.: IEEE 802.11-04/1009 September 2004 Abstract Introduction What is Performance “Prediction”? Different Categories of Prediction How Does TGT Fit? Conclusion and Suggested Focus Roger Skidmore Pratik Mehta (Dell), et al

September 2004 Introduction One of the goals of TGT is to enable prediction of performance Effectively, anyone who needs to predict 802.11 performance is in TGT’s audience Problem: How do we enable prediction of performance? Prediction means different things to different people What does it mean to predict performance? The purpose of this presentation is to take the first step toward how TGT can assist those who need to do predictions Roger Skidmore

What is Performance “Prediction”? September 2004 What is Performance “Prediction”? Prediction means different things to different people Is “prediction” a form of analysis, simulation, or something else entirely? Depends on the level of comfort and confidence of the user Most engineers are comfortable with measurements because they are “real” and “repeatable” and “mean something”, whereas predictions are viewed as dealing with “uncertainty” This is valid to a limited degree It is important to consider that people make all kinds of predictions every day For example, consider that you can not deploy an 802.11 network without making myriad assumptions and “predictions” A decision must be made about what to buy, where to put it, and how to configure it Note that measurements taken during the design phase may not reflect the “real” situation once the network is live, and may also not be very “repeatable” depending on the environment Roger Skidmore

What is Performance “Prediction”? (cont.) September 2004 What is Performance “Prediction”? (cont.) Performance prediction is a method of analysis that combines assumptions with accepted facts using algorithms and/or logic in order to reach a conclusion regarding the expected behavior of a device or group of devices under study The benefit is that (potentially) untenable problems are simplified and conclusions reached more quickly that (hopefully) fall within an acceptable margin of error Note that there is another layer of complexity and diversity in the types of predictions one can perform For example, what is a device? Am I planning a network, or building a better access point? Roger Skidmore

Different Categories of Predictions September 2004 Different Categories of Predictions Performance predictions can be categorized many ways based on what is being predicted By layer (e.g., MAC performance, PHY performance, etc.) Site-specific vs. non-site-specific Physical network/device vs. logical network/device Combinations of the above The person carrying out the prediction could have multiple goals, and these may even conflict For example, minimize number of APs but maximize coverage It is outside the scope of this presentation to catalog all the different types of performance predictions currently in use Instead, look for commonalities across major categories Roger Skidmore

Commonalities Across Predictions September 2004 Commonalities Across Predictions Combining assumptions and accepted facts with an algorithm and/or logic to reach a conclusion For the most part, Garbage in => Garbage out Remember that the goal is to reach a conclusion with an acceptable margin of error Assumptions Accepted Facts Algorithm and/or Logic Conclusion Roger Skidmore

September 2004 How Does TGT Help? TGT can help improve performance predictions in several ways: Providing increased number of Accepted Facts E.g., How does a device respond when placed under certain conditions? Providing increased number of valid Assumptions E.g., Is a device capable of supporting a certain traffic load? Providing increased confidence in the validity of Assumptions E.g., How has a device performed under similar tests conditions? Providing conclusions that can be used to empirically tune the Algorithm and/or refine the Logic (i.e., calibrate the prediction) E.g., Device A produced this output when under a certain set of conditions/inputs. Reproduce those conditions/inputs in the performance prediction and tune the algorithm so that the predictive analysis matches the measured result as closely as possible. The end result is a more accurate prediction of performance Roger Skidmore

Conclusion and Suggested Focus September 2004 Conclusion and Suggested Focus Most performance predictions are done using software today If TGT specified standard test conditions, methodologies, and a standard reporting format that could be easily parsed in software, many things can be automated A specification for conducting and reporting tests across a range of variable inputs would allow for more precise device characterization The level of the test (e.g., What layer? What type of device?) will determine what subset of performance predictions can utilize the data For example, a battery life test may not contain data directly beneficial to an analysis of an access point’s coverage area It will be impossible to address all possible tests – identify key tests with the most direct benefit across all of TGT’s audience Even standardized generic tests will still be beneficial to performance predictions Roger Skidmore