ELEKSPOT: EVALUATION PLAN Minkyu Lee

Agenda  Project Goal  Objective of Evaluation  Case Study: OpenStreetMap  Quality of GI  Phases of Evaluation

Project Goal  Project Goal  Enabling mobile devices to recognize indoor urban places based on Wi-Fi signal strength.  Question  How to cover very large amount of urban places?  Answer  Key is to build urban radiomap by citizen participation

Objective of Evaluation  There are many factors to evaluate…  What we want to show in this research is…  Then, we have to evaluate that It is possible to build city-wide radiomap by citizen participation (crowdsourcing) How crowdsourced urban radiomap is good

Case Study: OpenStreetMap  OpenStreetMap Overview  To build free street map for major cities by crowdsourcing based on wiki.

Case Study: OpenStreetMap  The 5 steps to making a map  Collect Data  Upload Data  Create/Edit OSM data  Label Data, and Add details  Render and Use Map!

Case Study: OpenStreetMap  Mapping party  To encourage and help people to edit map  Experienced mappers initiates

Case Study: OpenStreetMap  Evaluation of OpenStreetMap  M. Haklay, “How good is OpenStreetMap information? A comparative study of OpenStreetMap and Ordnance Survey datasets for London and the rest of England”  Ordnance Survey vs. OpenStreetMap  Comparison Factors Positional accuracy Completeness

Quality of GI (ISO 19113,19114)  Lineage  Positional accuracy  Attribute accuracy  Logical consistency  Completeness  Semantic accuracy  Usage, purpose and constraints  Temporal quality

Phases of Evaluation  Planning  Schedule, Budget, Area selection, …  Data Collection  Urban Radiomapping Workshop  Data Analysis  Accuracy  Completeness

Planning  Selected Area  Seoul, Gangnam-gu, Samseong-dong Building: 3200 Room: Public Space: 914

Planning  Schedule  Workshops About 200 places can be collected at a workshop Period : ~ Workshop: 5 times (once/twice a week) Participants: 5 ~ 10 persons  Budget  Devices 5 more T-mobile G1 devices. (60 만원 *5= 약 300 만원 )  Workshops 5 Workshops (10 명 *7 만원 (1 인 )*5 회 = 약 350 만원 )

Data Collection  Workshop Agenda  Project overview (20 min.)  Usage training and guide (40 min.)  Data collection (6 hrs)  Data upload and editing (40 min.)  Questionnaire (20 min.)  Eating and drinking! (~)

Data Collection  Target  Public spaces only (café, shop, theater, …)  Room-level  Mandatory Data  Place ID  Name  Place Scale  Latitude/Longitude  Fingerprints  Picture  Tags

Data Collection  Recruiting volunteers  Lab. members  Recruit in KAIST community (ara, eye)  Recruit in Android developer community

Data Analysis  Accuracy  Positional accuracy Precision Error distance (logical distance)  Attribute accuracy Room-levelFloor-levelBuilding-level 92.3%98.6%99.7% ExactAdj. roomSame floorSame bldg.Other bldg. 92.3%5.2%2%0.5%0% Place IDNameScaleTagsPosition 88%91.8%93%96.2%91.2% example

Data Analysis  Accuracy  Test dataset Collect test dataset after data collection Select randomly 300 public places in samseong-dong Collect 10 fingerprints for each place Conduct experiment in laboratory

Data Analysis  Completeness  Region completeness (samseong-dong) Building (%) – visualize on Google map Room (%)  Public space completeness (semseong-dong) Compare against public places in Daum Theme Map. Completeness (%)  Completeness of specific building (COEX) Room-coverage (%)

Data Analysis  Etc. (Statistics)  Averages APs in samseong-dong  Number of APs for each tag (café, shop, theater, …)  Number of contributed places for each person  …

Q&A  Thank you for listening