doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide TGn Simulation Methodology Special Committee March 2004 Report Jeff Gilbert Chair TGn SMSC Atheros Communications

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 2 History –Simulation methodology special committee was formed based on the belief by the majority of the TGn body that the modeling the PHY in MAC / System simulations should be examined and a mandatory or recommended (TBD) methodology should be developed –Issue was raised: J.Gilbert (Atheros) – Albuquerque Large consensus in 1/6/04 FRCC telecon. of need H.Bonneville and B.Jechoux (Mitsubishi) Vancouver – 1/15/04 TGn straw polls and votes

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 3 Charter –Simulation Methodology Special Committee was discussed and approved in the TGn session on 1/15/04 and announced at the 1/16/04 closing plenary –Three bi-weekly conference calls between the January and March meetings were held –Special Committee chartered through March Plenary (w/ extension by TGn vote if nec.) to define a PHY-MAC interface to be used in MAC simulations generating results reported in the CC matrix –Following the sub-committee's completion there will be a TGn vote to determine whether the methodology will be mandatory or optional

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 4 Relationship of SMSC to FRCC The Simulation Methodologies discussed in the context of the SMSC are relevant to the MAC/System simulations (not the PHY-only ones) The affected Comparison Criteria would be : – Performance measurements at the MAC Sap – CC15 - Sharing of medium with legacy devices.

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 5 PHY Abstraction problem –PHY / MAC interface can dramatically impact overall results: Time varying channel creates time varying PER Affects overall delay, jitter, and throughput PHY / MAC rate adaptation effects performance –Challenges Properly model channel and PHY characteristics in MAC sims Properly model interactions between PHY and MAC Keep flexibility to readily adapt to different proposals’ PHYs Keep simulation effort reasonable

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 6 Risks of not specifying Abstraction –Without common PHY/MAC interface, results could depend more on simulation methodology than proposal –Possible interpretations: Idealized PHY Distance -> mean SNR (w, w/o interference) -> PER Distance -> fading SNR (w, w/o interference) -> PER Distance -> fading SNRs -> capacity formula -> PER Fully accurate PHY model w/rate adaptation -> PER lookup Approximate PHY based on SNR profile in MAC simulation Full PHY simulation per packet in MAC simulation –Standardizing the abstraction could allow for more meaningful comparisons and save significant time during the proposal evaluation selection process

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 7 Goals Goals of a common PHY / MAC interface: –Allow fair comparison of MAC / System results –Facilitate verification / cross-checking of results Intermediate results to simplify process –Possible other goals: Ability to merge MAC and PHY from different proposers Ability to simulate MAC-only modifications with known PHY

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 8 Goals of the SMSC –Determine a methodology that the TGn body would choose to adopt to allow fair comparisons –If a single methodology cannot be determined, refine 2-3 to be selected from at the March 2004 session –All members of special committee should provide positive feedback to improve the various approaches and focus on consensus

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 9 Process of the SMSC –It was critical to move quickly in order to have conclusions to present at the March 2004 session –The were 3 calls prior to the March 2004 session: Feb 05, :00 Pacific Time Feb 19, :00 Pacific Time Mar 04, :00 Pacific Time –Overall agenda of calls: During the 1 st call, the goal of the group and overview of different approaches were discussed During the 2 nd call, specific proposed methodologies were discussed During the 3 rd call, merged proposal(s) to be presented to the TGn body were discussed in more detail

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 10 Results of SMSC Telecons –Simulation methodology requirements, sample TGn technologies, and simulation tools discussed –Several methodologies discussed (good discussions etc.) From Atheros, Intel, Marvell, Mitsubishi, Qualcomm, ST Micro Methodologies merged down to two groups: Intel/Qualcomm and Atheros/Marvell/Mitsubishi/ST Micro –Commonalities Base PHY performance in MAC model on TGn channel model –Differences Simplified PHY vs. Simplified PHY/MAC interface –Consensus was not reached between the two groups The topic of mandatory vs. optional was out of the group’s scope, though there were differing opinions

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 11 Methodology Requirements –These requirements were discussed on the first conference call and updated on the calls and/or s. –These have not been ranked / approved by the group. The requirements are to accurately model: Bit / packet errors Channel variation and resulting varying PER Rate adaptation interactions with the PHY PHY impairments Beamforming gains Interference effects MIMO MAC operation with multiple independent data streams –Additional metric for comparison is compute resource requirements: CPU cycles and storage

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 12 Sample TGn Technologies –By considering the variety of technologies to be proposed for TGn, the methodologies can make sure that they can adequately represent performance. –This list was discussed on the first call and will be updated as new suggestions come on the calls or s. –These have not been ranked / approved by the group: Adaptive bit-loading –Closed loop –Many more coding rates Beamforming / nulling LDPC / other coding methods MIMO / Multiple data streams

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 13 Simulation Tools Used –This page lists the simulation tools used by members. This is informative only and does not imply that a particular tool or tools are mandated or recommended by the SMSC –This list was discussed on the 1st conference call and updated from new information on the calls and s: Opnet (7): Intel, Mitsubishi, Nortel, NTT, Samsung Electronics, TI, Toshiba MLDesigner (1): Caller not present on second call NS (6): Airgo, Atheros, Fujitsu, Marvell, Qualcomm, ST Microelectronics QualNet: UCLA

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 14 Methodology Options Discussed –Black-box PHY methods (11-04/0172 -Mitsubishi /Atheros) Use full accuracy of PHY sims w/ rate adapt encapsulated in tables Tight coupling of channel & PHY models, looser coupling w/ MAC –PHY integrated into MAC methods (11-03/ Intel) Parametric version of PHY incorporated in system simulations Looser coupling of channel & PHY models, tighter coupling w/ MAC –“Channel Capacity to determine PERs” method (11-04/0064 STM) Markov model for channel capacity mapped to PERs Channel capacity links Markov channel model and per-rate PERs –Record and Playback PHY Abstraction (11-04/ Marvell) Record sequences in PHY sims w/rate adapt and play back in MAC sims –TGn Sim Methodology Validation Proposal (11-04/0185 Atheros) Simplified validation for point-to-point case with full PHY/MAC

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 15 Merged Methodologies –Unified black-box PHY methods using Channel Capacity Atheros / Mitsubishi / ST / Marvell Use full accuracy of PHY sims w/ rate adapt encapsulated in tables Tight coupling of channel & PHY models, looser coupling w/ MAC Real channel model run in MAC simulation –Channel Capacity used to index look-up tables. Described in document 11-04/0218 –PHY Abstraction for System Simulation Parametric version of PHY incorporated in system simulations Real channel model run in MAC simulation PHY / MAC interface is exact Particular approximation of PHY will vary by proposal Described in documents 11-03/0863 and 11-04/0174

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 16 Pros & Cons of Mandating a Simulation Methodology Cons Constrains proposers to a methodology that may not be their top choice Takes more time now to select and finalize a methodology There are currently some open issues with the proposed methods Pros Would make comparison of MAC/System results more meaningful Could reduce time debating methodologies & results later (eg a) If different proposals use different methods, the impact could be amplified

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 17 Questions for TGn –Is it desirable to standardize the simulation methodology detailing how the PHY is incorporated into MAC / System simulations? –If so, are any of the approaches described so far acceptable? Which one(s)? –If a full methodology for MAC / System simulation cannot be standardized, is it beneficial to standardize a more limited PHY / MAC test?

doc.: IEEE /0301r0 Submission March 2004 J.Gilbert, Atheros CommunicationsSlide 18 Plans for the March Session Presentations –This overview –Presentations from two main proposals –Other supporting presentations Discussion & Straw Polls Votes