1. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 1 Project: IEEE P802.15 Working Group for Wireless Personal Area Networks (WPANs) Submission Title: [TG3 Common Simulation Framework Presentation] Date Submitted: [July 9.2000] Source: [P. Taylor Salman, Manager, Market Research] Company [OPNET Technologies, Inc] Address [3400 International Drive, Washington, DC 2000] Voice:[202-364-4700x2297], FAX: [Add FAX number], E-Mail:[tsalman@mil3.com] Re: [.] Abstract:[ The topic is a common simulation framework (CSF). A CSF allows data to be collected on competing standards proposals with identical usage scenarios, traffic loads and types, interfering sources, and methods of statistical collection. The goal is to be able to compare proposals, apples to apples, and to arrive at an objective conclusion on the best technical implementation. While other issues still play into the standards development process, CSF implementation can offer an approach to this task. ] Purpose:[ The purpose of this presentation is to describe the benefits of implementing a CSF, how to best implement a CSF, what issues can be addressed by modeling and simulation in the standards process, and a review of past and present efforts within IEEE 802 using modeling and simulation..] Notice:This document has been prepared to assist the IEEE P802.15. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein. Release:The contributor acknowledges and accepts that this contribution becomes the property of IEEE and may be made publicly available by P802.15.

2. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 2 P. Taylor Salman, Manager, Market Research Using Modeling and Simulation to Support Communications Standard Development

3. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 3 Outline Introduction Modeling & Simulation? Process, Pitfalls, & Lessons Learned Common Applications of M&S in the Standard Development Process How To… Common Simulation Framework Coexistence Studies Standards Enhancement Summary

4. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 4 Introduction One of the biggest challenges in developing successful standards is determining the accuracy and performance of competing proposals. Subjective rating systems and unsubstantiated claims can inhibit progress and sometimes lead to stalemate within a working group. Modeling and simulation of proposals creates the opportunity for objective comparison and clear validation of proposals before final decisions are made.

5. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 5 Introduction How does M&S improve the standards development process? What is the best process for using M&S? What pitfalls should be avoided? When should modeling and simulation be used? How should M&S be applied?

6. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 6 Modeling and Simulation? Networking technology has become too complex for traditional analytical methods to yield accurate understanding of system behavior

7. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 7 Modeling and Simulation? Addresses multiple layers of performance Architecture Application Transport&Network Access Physical Divides complex issues into understandable and resolvable components

8. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 8 Modeling and Simulation? Provides hard data from a variety of perspectives to evaluate performance Graphic depiction of performance eases communication of concepts and proofs

9. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 9 The Process Scope the ProblemDivide & ConquerFormat the Answer Design ScenariosBuild ModelsRun Simulations Compare Results

10. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 10 The Process Scope the Problem –What questions should be answered to evaluate the proposals? –5 Criteria –Translate into data measures (MOEs) Divide & Conquer –Split the questions into smaller problems –Group like issues –Isolate desired behavior –Abstract unrelated behavior Format the Answer –What data will support the MOEs? –Where should the data be collected? –What method should be used to collect the data? –How should the data be presented?

11. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 11 The Process Design Scenarios –Determine the most likely conditions for use –Determine extreme environments –Design multiple scenarios to address both Type of approach –Analytic vs. discrete-event, combination –Bit level vs. packet level vs. flow level performance Topology Applications/traffic sources Higher-layer protocol interfaces Simulation run-time parameters Statistical distributions –Limit the scope of the scenario to address desired behavior –Determine how scenarios can “feed” each other

12. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 12 The Process Build Models –Data, Data, Data… –Document! –Share and vet… Run Simulations –New Issues??? –Improve the process… Compare Results –Data will NOT provide the answer… –Analysis is needed, arguments necessary… –Proof is available, arguments are objective and verifiable Performance is just one of many factors contributing to a formal standard; M&S helps us understand the implications of our performance tradeoffs…

13. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 13 Pitfalls & Lessons Learned Start Early! Get Commitments Divide Responsibility Communicate –Share models Get agreement before proceeding Re-evaluate

14. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 14 Applications of M&S in the Standards Process Common Simulation Framework –Test different proposals for a common objective –Uses commons scenarios, traffic sources, higher layer protocols, data collection methods, and run time parameters –Enables “Apples to Apples” comparison Co-existence Studies –Evaluate effect of new standards on existing systems –Physical and higher layer performance –Create complex environments without prototyping Protocol Enhancement –Improve or “add on” to an existing protocol –Add features without affecting current performance –Test integration in a virtual environment

15. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 15 Examples of M&S Applications in the Standards Process DOCSIS CSF (CableLabs) Bluetooth Co-existence Wireless LAN Enhancement

16. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 16 Summary Modeling & Simulation provides verifiability and objectivity to the standards development process Commitment, a plan, and adherence to a process are necessary to be successful Don’t “bite off more than you can chew” and don’t succumb to the “ever-expanding scope”! Additional Benefits: –The final models provide a common understanding of the standard –Models can be leveraged: –Promotion of the technology –Architecture planning –2 nd generation technology baseline

17. doc.: IEEE 802.15-00/223r0 Submission July 2000 Taylor Salman, Opnet TechnologiesSlide 17