CSRIC WG#3 Indoor Location Accuracy Test Bed Report March6, 2013 Stephen J. Wisely & Richard Craig Co-Chairs

Outdoor Accuracy Testing Indoor Accuracy Testing Impact of Commercial LBS Technologies 2 Work Group 3 Areas of Focus

 The WG addressed the FCC’s questions as best we could for now  We developed a testbed framework and have solicited interest from suppliers of current technology on their in interest in having their product go through the TB  We are currently working on the finer points of the testing process and logistics associated with TB program oversight  We anticipate conducting testing in two stages:  Stage I – Characterize performance of current technologies  Stage II – review of future technologies  We will provide a read out on Stage I efforts to the CSRIC committee at the September 2012 and March 2013 meetings  We believe the Testbed concept could survive beyond the current CSRIC charter as an independently supported entity for evaluating future technologies 3

Participants 4 First NameLast NameOrganization BrettSchneiderBexar Metro Network District DeWayneSennettATIS NormanShawPolaris Wireless, Inc. SusanSherwoodVerizon Wireless GirishSivaramTeleCommunications Systems, Inc. (TSC) JohnSnappIntrado, Inc. DorothySpears-DeanVirginia Information Technologies Agency BillTortorielloUS Cellular GregTuretzkyCSR Technology Inc. BruceWilsonQualcomm Inc. StephenWiselyAPCO International Richard Deh- Min WuNokia Siemens Networks WG 3 Indoor Location Test Bed Subgroup consisted of the following members: First NameLast NameOrganization WayneBallantyneMotorola Mobility, Inc. AndrewBeckCommScope RichardCraigVerizon Wireless KhaledDessoukyTechnoCom Corporation ThomasDombrowskyWiley Rein LLP JeannaGreenSprint RogerHixsonNENA RyanJensenT-Mobile MarteKinderTime Warner Cable SandraLottCenturyLink MikeLoushineApplied Communication Sciences BarryMartinBoeing KathyMcMahonAPCO International MartinMoodyMetro Emergency Services Board GaryParsons NextNav LLC GaneshPattabiranan NextNav LLC GustavoPavonTrue Position, Inc. RaghavendhraRaoAT&T ChuckRonshagenCassidian Communications

Indoor Test Bed –Stage 1 Goal: Independent comparative testing that provides the FCC with meaningful, objective results on indoor wireless location performance Constraints: Within time frame of CSRIC III; financially feasible framework WG3 chose San Francisco Bay Area due to the wide range of wireless environments (morphologies) present Invitation extended to a host of location technology vendors to participate 7 vendors started the process and 3 eventually participated Two existing and one new location technologies tested: – NextNav (wireless beacon technology) – Polaris Wireless (RF pattern matching/fingerprinting) – Qualcomm (AGPS/AFLT hybrid) Four major wireless carriers participated: – AT&T, Sprint, T-Mobile, Verizon Competent, experienced independent test house selected (TechnoCom) 5

Indoor Test bed Methodology Based on sound statistical test methodology – Articulated in ATIS ; received wide consensus Multiple buildings (19) of different sizes & types across range of environments – Dense urban, urban, suburban, and rural morphologies Multiple test points in each building – 2 to 6 test points depending on size and complexity of selected building – Representing typical use scenarios Statistically significant, adequately large number of independent test calls at each stationary test point – 100 test calls per technology minimum target per test point – >13,400 test calls per technology placed at 74 test points, well over 100 avg. – Aggregated over different devices and networks used by the location technology under test at each test point Indoor ground truth accuracy < 1 m (professional survey accuracy 2 cm) Performance attributes tested: accuracy, yield, TTFF, uncertainty, scatter 6

Test Environments and Buildings 7 Dense urban Downtown Financial District San Francisco (6 bldgs) Urban Downtown San Francisco and San Jose (5 bldgs) Suburban Santa Clara and Sunnyvale (6 bldgs) Rural San Benito County (2 bldgs)

Urban environment had the most challenged horizontal accuracy followed closely by dense urban Summary Results-- Accuracy 8 NextNav technology provided vertical results

Summary Results: Yield, TTFF, and Uncertainty Yield varied with the severity of the environment but was generally high – >90% for all technologies in the configurations tested with one minor exception (86% AGPS/AFLT yield in dense urban) – More yield variation for the technology actually integrated into the wireless network and has to exchange messages per the standard E911 implementation through severe RF fading TTFF was stable and not an issue – Small variation about design values for non-network integrated implantations 27 sec. for NextNav and 24 sec. for Polaris – More variation for AGPS/AFLT, which is actually integrated into the network 28 sec. avg. with 33 sec. 90% in urban/dense urban vs. 24 sec avg. & 26 sec. 90% suburban/rural Uncertainty is still a useful parameter and its reliability generally reflects how well a system is performing in a given environment 9

Location Scatter Relative to Building Sizes & Density Traditional benchmarks from outdoor performance may not best suite the indoors, especially in denser cities Even traditionally good performance may not meet the unmitigated expectations of some--education on indoor issues is important 10 Test points in the 5-story building partially shadowed by tall bldg Test point 3 (T3) at center of 50/150 m circles

Lessons and Recommendations The cooperatively funded indoor test bed is a viable framework, which yields very valuable results – It’s a model that should be repeated with some adjustments, based on the lessons learned – Key to its success are the experience & strong commitment of its participants  Create and maintain the CSRIC-type framework that fosters the active and balanced participation by the various stakeholders There are a number of difficult challenges that need to be met repeatedly – Project setup and contractual arrangements are difficult and time consuming – Building access is the most vexing of challenges facing indoor testing  Allow for the long lead time of several months required to properly tackle these issues  Create a test management resource with contractual authority and some funding capability  Develop a more structured approach to dealing with building access and, as needed, use creative incentive mechanisms for buildings to participate Deployed location technologies help with indoor performance and new technologies tested have promising performance, but there is a long way to go to meet expectations for reliable indoor use of wireless location for E911  Continually test new technologies, including next generations of existing systems, technologies used for LBS (e.g., WiFi) & new technologies targeted to the latest air interfaces The Bay Area has proven to be a good place for the indoor test bed  Leverage it fully in future test beds 11