802.11 TGn Simulation Methodology Special Committee March 2004 Report Month 2004 doc.: IEEE 802.11-03/0170r2 802.11 TGn Simulation Methodology Special Committee March 2004 Report Jeff Gilbert Chair TGn SMSC Atheros Communications Jeff Gilbert: Specify that we did boil down to 2 proposals Consensus was not reached between the two Consensus was not reached on mandatory / optional Goal of methodology is to compare different proposals with different phy and mac If there is variability/arguments now, worse later Make clear that those effected parts of the FRCC are only:     4.4.1 - Performance measurements at the MAC Sap    and CC15 - Sharing of medium with legacy devices.  J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Month 2004 doc.: IEEE 802.11-03/0170r2 History Simulation methodology special committee was formed based on the belief by the majority of the TGn body that the modeling the PHY in MAC / System simulations should be examined and a mandatory or recommended (TBD) methodology should be developed Issue was raised: J.Gilbert (Atheros) - 11-03-0888 – Albuquerque Large consensus in 1/6/04 FRCC telecon. of need H.Bonneville and B.Jechoux (Mitsubishi) 11-04-0120 Vancouver – 1/15/04 TGn straw polls and votes J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Month 2004 doc.: IEEE 802.11-03/0170r2 Charter Simulation Methodology Special Committee was discussed and approved in the TGn session on 1/15/04 and announced at the 1/16/04 closing plenary Three bi-weekly conference calls between the January and March meetings were held Special Committee chartered through March Plenary (w/ extension by TGn vote if nec.) to define a PHY-MAC interface to be used in MAC simulations generating results reported in the CC matrix Following the sub-committee's completion there will be a TGn vote to determine whether the methodology will be mandatory or optional J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Relationship of SMSC to FRCC The Simulation Methodologies discussed in the context of the SMSC are relevant to the MAC/System simulations (not the PHY-only ones) The affected Comparison Criteria would be : 4.4.1 - Performance measurements at the MAC Sap CC15 - Sharing of medium with legacy devices. J.Gilbert, Atheros Communications

PHY Abstraction problem Month 2004 doc.: IEEE 802.11-03/0170r2 PHY Abstraction problem PHY / MAC interface can dramatically impact overall results: Time varying channel creates time varying PER Affects overall delay, jitter, and throughput PHY / MAC rate adaptation effects performance Challenges Properly model channel and PHY characteristics in MAC sims Properly model interactions between PHY and MAC Keep flexibility to readily adapt to different proposals’ PHYs Keep simulation effort reasonable J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Risks of not specifying Abstraction Month 2004 doc.: IEEE 802.11-03/0170r2 Risks of not specifying Abstraction Without common PHY/MAC interface, results could depend more on simulation methodology than proposal Possible interpretations: Idealized PHY Distance -> mean SNR (w, w/o interference) -> PER Distance -> fading SNR (w, w/o interference) -> PER Distance -> fading SNRs -> capacity formula -> PER Fully accurate PHY model w/rate adaptation -> PER lookup Approximate PHY based on SNR profile in MAC simulation Full PHY simulation per packet in MAC simulation Standardizing the abstraction could allow for more meaningful comparisons and save significant time during the proposal evaluation selection process J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Goals Goals of a common PHY / MAC interface: Month 2004 doc.: IEEE 802.11-03/0170r2 Goals Goals of a common PHY / MAC interface: Allow fair comparison of MAC / System results Facilitate verification / cross-checking of results Intermediate results to simplify process Possible other goals: Ability to merge MAC and PHY from different proposers Ability to simulate MAC-only modifications with known PHY J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Month 2004 doc.: IEEE 802.11-03/0170r2 Goals of the SMSC Determine a methodology that the TGn body would choose to adopt to allow fair comparisons If a single methodology cannot be determined, refine 2-3 to be selected from at the March 2004 session All members of special committee should provide positive feedback to improve the various approaches and focus on consensus J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Month 2004 doc.: IEEE 802.11-03/0170r2 Process of the SMSC It was critical to move quickly in order to have conclusions to present at the March 2004 session The were 3 calls prior to the March 2004 session: Feb 05, 2004 08:00 Pacific Time Feb 19, 2004 08:00 Pacific Time Mar 04, 2004 08:00 Pacific Time Overall agenda of calls: During the 1st call, the goal of the group and overview of different approaches were discussed During the 2nd call, specific proposed methodologies were discussed During the 3rd call, merged proposal(s) to be presented to the TGn body were discussed in more detail J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Results of SMSC Telecons Month 2004 doc.: IEEE 802.11-03/0170r2 Results of SMSC Telecons Simulation methodology requirements, sample TGn technologies, and simulation tools discussed Several methodologies discussed (good discussions etc.) From Atheros, Intel, Marvell, Mitsubishi, Qualcomm, ST Micro Methodologies merged down to two groups: Intel/Qualcomm and Atheros/Marvell/Mitsubishi/ST Micro Commonalities Base PHY performance in MAC model on TGn channel model Differences Simplified PHY vs. Simplified PHY/MAC interface Consensus was not reached between the two groups The topic of mandatory vs. optional was out of the group’s scope, though there were differing opinions J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Methodology Requirements Month 2004 doc.: IEEE 802.11-03/0170r2 Methodology Requirements These requirements were discussed on the first conference call and updated on the calls and/or emails. These have not been ranked / approved by the group. The requirements are to accurately model: Bit / packet errors Channel variation and resulting varying PER Rate adaptation interactions with the PHY PHY impairments Beamforming gains Interference effects MIMO MAC operation with multiple independent data streams Additional metric for comparison is compute resource requirements: CPU cycles and storage J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Sample TGn Technologies Month 2004 doc.: IEEE 802.11-03/0170r2 Sample TGn Technologies By considering the variety of technologies to be proposed for TGn, the methodologies can make sure that they can adequately represent performance. This list was discussed on the first call and will be updated as new suggestions come on the calls or emails. These have not been ranked / approved by the group: Adaptive bit-loading Closed loop Many more coding rates Beamforming / nulling LDPC / other coding methods MIMO / Multiple data streams J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Month 2004 doc.: IEEE 802.11-03/0170r2 Simulation Tools Used This page lists the simulation tools used by members. This is informative only and does not imply that a particular tool or tools are mandated or recommended by the SMSC This list was discussed on the 1st conference call and updated from new information on the calls and emails: Opnet (7): Intel, Mitsubishi, Nortel, NTT, Samsung Electronics, TI, Toshiba MLDesigner (1): Caller not present on second call NS (6): Airgo, Atheros, Fujitsu, Marvell, Qualcomm, ST Microelectronics QualNet: UCLA J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Methodology Options Discussed Month 2004 doc.: IEEE 802.11-03/0170r2 Methodology Options Discussed Black-box PHY methods (11-04/0172 -Mitsubishi /Atheros) Use full accuracy of PHY sims w/ rate adapt encapsulated in tables Tight coupling of channel & PHY models, looser coupling w/ MAC PHY integrated into MAC methods (11-03/0863 - Intel) Parametric version of PHY incorporated in system simulations Looser coupling of channel & PHY models, tighter coupling w/ MAC PHY Abstraction for System Sim (11-04/0174 – Qualcomm/Intel) Run TGn channel models with PHY abstraction under MAC sims Space-frequency channel & interference explicitly captured in sims “Channel Capacity to determine PERs” method (11-04/0064 STM) Markov model for channel capacity mapped to PERs Channel capacity links Markov channel model and per-rate PERs Record and Playback PHY Abstraction (11-04/0183 - Marvell) Record sequences in PHY sims w/rate adapt and play back in MAC sims TGn Sim Methodology Validation Proposal (11-04/0185 Atheros) Simplified validation for point-to-point case with full PHY/MAC J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Month 2004 doc.: IEEE 802.11-03/0170r2 Merged Methodologies Unified black-box PHY methods using Channel Capacity Atheros / Mitsubishi / ST / Marvell Use full accuracy of PHY sims w/ rate adapt encapsulated in tables Tight coupling of channel & PHY models, looser coupling w/ MAC Real channel model run in MAC simulation Channel Capacity used to index look-up tables. Described in document 11-04/0218 PHY Abstraction for System Simulation Parametric version of PHY incorporated in system simulations PHY / MAC interface is exact Particular approximation of PHY will vary by proposal Described in documents 11-03/0863 and 11-04/0174 J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Pros & Cons of Mandating a Simulation Methodology Constrains proposers to a methodology that may not be their top choice Takes more time now to select and finalize a methodology There are currently some open issues with the proposed methods Pros Would make comparison of MAC/System results more meaningful Could reduce time debating methodologies & results later (eg 802.15.3a) If different proposals use different methods, the impact could be amplified J.Gilbert, Atheros Communications

Month 2004 doc.: IEEE 802.11-03/0170r2 Questions for TGn Is it desirable to standardize the simulation methodology detailing how the PHY is incorporated into MAC / System simulations? If so, are any of the approaches described so far acceptable? Which one(s)? If a full methodology for MAC / System simulation cannot be standardized, is it beneficial to standardize a more limited PHY / MAC test? J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications

Plans for the March Session Month 2004 doc.: IEEE 802.11-03/0170r2 Plans for the March Session Presentations This overview Presentations from two main proposals Other supporting presentations Discussion & Straw Polls Votes J.Gilbert, Atheros Communications J.Gilbert, Atheros Communications