1. Location-based search: services, photos, web Andrei Tabarcea Mohammad Rezaei 4.12.2013

2. Introduction The goal is to find services, photos and points of interest close to the user’s location We call this “location-based search” We try to search our local database of photos and services and to find location information in web-pages keyword Results on map User location

3. MOPSI search Mopsi Services Database Mopsi Photo Collection Mopsi Web Search Combinationofsearchresults User location keyword Mopsi search

4. Web interface Input: (keyword, user location) Output: array of results keyword Results list Search options Results on map User location

5. Mobile interface User location Search results

6. Mopsi search (server workflow) Input: (keyword, flagS, flagP, flagW, user location) Output: (g_markersData) array of results keyword Results list Search options flagS flagP flagW Results on map User location

7. Location based search Input: keyword, flagS, flagP, user location (lat,lon) Output: list of results Note: A service has a list of keywords and a title A photo has just a description So, Keyword search is done according to this information Notation: S: service, P: photo text(S): keywords and title of service text(P): description of photo flagS: search for services if true flagP: search for photos if true

8. Overall flow Start Update keywords statistics Update keywords history flagS Y Photo search and add results to the list flagP flagW Local service search and display results in list Web search Add results to the list and on map End Y Y N N N When a keyword is searched: statistics: the count of it in database is incremented, keyword and city are stored history: keyword, location, userid and time are stored Stage 1: Search mopsi services Display all results on Map Stage 2: Search mopsi photos Stage 3: Search web

9. Local service search Start nL>0 Do search on server nL=number of results Display results in the list Y N End Cluster results with almost same title and location Sort the results (distance to user location) Take and display one of the similar results as representative The list of results

10. Photo search Start nP>0 Do search on server nP=number of results Y N End Cluster results with almost same title and location Cluster the results and Local services with almost same title and location Sort the results (distance to user location) Add results to the list

11. Web search Start nW>0 Do search on server nW=number of results Y N End Cluster results with almost same title and location Cluster the results and Local services and photos with almost same title and location Sort the results (distance to user location) Add results to the list Add results on the map

12. Filtering results: old solution Fixed distance to user location: d Find services where text(S) ≈ keyword AND dist(S,User) < d Find photos where text(P) ≈ keyword AND dist(P,User) < d d Advantages: Simple Same time for any search Disdvantages: Parameter d (User can choose d, but still not automatic) There are many cases with “no results”

13. Current solution: Binary search K-nearest services Show all the results in 10 km If number of results is less than K, double the distance (until whole earth), when number of results is bigger than K, divide the distance d 2d 4d x Example with k=5: Number of results n in distance d: 1 < k Double distance: in 2d, n=2 < k In 4d, n=8 > k Now dividing distance in colored area: In 3d, n=4 < k In 3.5d, n=5 (=k) So, we have 5 nearest results to user location in distance x User location A photo or service with required keyword

14. Algorithm d=10000: initial distance K=10: number of required results delta_dist: minimum distance for dividing ns: number of resulted services res_S np: number of resulted photos res_P res_S = services where text(S) ≈ keyword res_P = photos where text(P) ≈ keyword if ( ns+np > K ) (res_S res_P dist) = extend_distance(); (res_S res_P dist) = contract_distance(); display (ns+np) services and photos extend_distance() ns= 0; np=0; While ( ns+np < K AND dist < earth_r*pi) res_S = services where text(S) ≈ keyword AND dist(S,User) < dist res_P = photos where text(P) ≈ keyword AND dist(P,User) < dist dist = dist*2 dist = dist/2 d 2d 4d Δ

15. Algorithm (cont.) contract_distance(dist, K) d1 = dist/2 d2 = dist dist = (d1 + d2)/2 delta = dist – d1 ns=np=0 While ( ns+np != K AND delta > delta_dist AND dist > d ) res_S = services where text(S) ≈ keyword AND dist(S,User) < dist res_P = photos where text(P) ≈ keyword AND dist(P,User) < dist if ( ns+np > K ) d1 = d1; d2= dist else d1 = dist; d2 = d2 dist = (d1 + d2)/2 delta = dist-d1

16. Simplifying distance calculation Since there is no spatial dist function in mysql: Points with distance < d from user location Simplified: |lat-lat 1 |< Δlat AND |lon-lon 1 |< Δlon User location (lat 1, lon 1 ) d d (lat 1 + Δlat, lon 1 ) (lat 1, lon 1 + Δlon) Δlat and Δlon? lat 1, lon 1 d (in meter )

17. Δlat and Δlon? Distance d (in meter) between two points (lat 1, lon 1 ) and (lat 2, lon 2 ): Earth diameter (in meter) Haversine distance: (lat 1, lon 1 ) and (lat 1, lon 1 + Δlon)  Δlat=0 (lat 1, lon 1 ) and (lat 1 + Δlat, lon 1 )  Δlon=0

18. How to find location-information in web-pages? Location-based web data mining

19. Mopsi web search Web mining

20. Geo-referencing Geo-referencing: A geographic reference is an information entity that is discovered from the context and can be mapped to a geographic location Strategies for geographic reference extraction: – Gazetteer-based text matching – Rule-based linguistic analysis – Regular-expression based text matching – Using host location – Geographic meta-tags Hu, Y. H., Lim, S., & Rizos, C. Georeferencing of Web Pages based on Context-Aware Conceptual Relationship Analysis. 2006

21. Ad-Hoc Georeferencing The problem is how to extract and validate location data from semi- structured text Postal address is the most common location data found Our goal is to give geographical coordinates to services mentioned in web-pages We call this method ad-hoc georeferencing Pages of Pasi Fränti VS.

22. Location Information in Webpages Site hosting information (owner address, server address etc.) HTML tags (geo-tags, address-tags, vcards for Google Maps etc.) Natural language descriptions Addresses, postal codes, phone numbers

23. Site hosting information domain: uef.fi descr: ITÄ-SUOMEN YLIOPISTO (UNIV OF EASTERN FINLAND) descr: 22857339 address: TIETOTEKNIIKKAKESKUS (IT-CENTRE)/Jarno Huuskonen address: PL 1627 address: 70211 address: KUOPIO FINLAND phone: +358 44 7162810 status: Granted created: 26.5.2010 modified: 19.8.2011 expires: 26.5.2015 nserver: ns-secondary.funet.fi [Ok] nserver: ns1.uef.fi [Ok] nserver: ns2.uef.fi [Ok] dnssec: no

24. geo-tags, address-tags, vcards for Google Maps etc. HTML tags Pages of Pasi Fränti

25. Natural language descriptions Scouts' Youth Hostel Scouts' Youth Hostel (8.3 km from Joensuu Airport ) Show map Show map Good, 7.4 Latest booking: January 23 Scouts’ Youth Hostel is located at the outfall of River Pielisjoki, 1.5 km from Joensuu city centre. It offers free Wi-Fi and rooms with shared bathroom and kitchen facilities. Olga Saint-Petersburg, Russia "Great price for the nice room. Friendly stuff, cozy atmosphere. But a bit loud." from € 46 € 46

26. Postal addresses

27. Input: user location (lat, lon) keywords Output: list of services containing: name/title website address (street, number. city) location (lat, lon) image other info (opening hours, telephone etc.) Main idea: preprocess the search results of an external search engine (Google, Yahoo, Bing etc.) by detecting postal address in order to find the location Mopsi search

28. Problems -How to evaluate relevance? -Mixed keyword meanings -No relation between keywords and addresses

29. Mopsi Web Search Workflow Geocoded street-name database Geo-referencing module Mobile application Web user interface Coordinates Address Keyword Coordinates Search results Keyword Coordinates Search results

30. Georeferencing module Geocoded database Address and description detector Address validator Word list Results list Sorted results list Keyword Municipalities query Result links Coordinates Municipalities list Addresses Coordinates Relevant municipalities detector Keyword, Address, Coordinates Page parser

31. 1.Convert user location (lat, lon) into user address = Geocoding step 2.Search with the query "keyword+city" using an external search engine API and download the first k results (web pages) = Web page retrieval step 3.Detect addresses and additional informatio from the downloaded web pages = Data mining step 4.Ranking the results (distance, relevance etc.) = Ranking step 5.Display the search results to the user Proposed steps 1. Geocoder 2. Web page retrieval 3. Data mining 4. Result ranking User lat, lon keywords web pages result list 5. ranked result list

32. 1. Geocoding Geocoder Web page retrieval Data mining Result ranking User lat, lon keywords web pages result list ranked result list Convert user location (lat, lon) into user address using:

33. 2.Web page retrieval Geocoder Web page retrieval Data mining Result ranking User lat, lon keywords web pages result list ranked result list Download k webpages from the query using API of:

34. 3.Data mining Geocoder Web page retrieval Data mining Result ranking User lat, lon keywords web pages result list ranked result list Main idea: Find location information in HTML pages by detecting postal addresses Steps: 1.Parse and segment the HTML page 2.Identify addresses and locations 3.Identify the services the addresses are pointing to (name/title) and retrieve extra information (photos, opening hours, telephone etc.)

35. 3.1 Parsing HTML pages -Current solution extracts an array of text from HTML pages -We don’t exploit the advantage that we extract data from web pages -Proposed future solution: -Segmentation of web pages using DOM trees -Detection of the address block -Nearest-neighbor search considering text and visual characteristics Joen Pizza Special Y-tunnus 2129577-6 Käyntiosoite Koskikatu 17 80100 JOENSUU Postiosoite Koskikatu 17 80100 JOENSUU Puhelin: 013-220246 Virallinen toimiala Kahvila-ravintolat

36. Web page example - Homepage

37. DOM tree blue: links (the A tag) red: tables (TABLE, TR and TD tags) green: dividers (DIV tag) violet: images (the IMG tag) yellow: forms (FORM, INPUT, TEXTAREA, SELECT and OPTION tags) orange: linebreaks and blockquotes (BR, P, and BLOCKQUOTE tags) black: HTML tag, the root node gray: all other tags

38. DOM subtree PizzaPojat Niinivaara Niinivaarantie 19 80200 Joensuu 013 - 137 017 PizzaPojat Niinivaara Niinivaarantie 19 80200 Joensuu 013 - 137 017

39. Web page example - Catalog Bosbor kebab Fiesta Miami

40. PizzaPojat Niinivaara Niinivaarantie 19 80200 Joensuu 013 - 137 017 1.Convert HTML pages to xHTML for using xQuery 2.Detect addresses and postal codes 3.Break the DOM tree into subtrees 4.Use heuristics and regular expressions to detect extra information from the subtree (service name, telephone, opening hours etc.) Proposed implementation

41. Rule-based pattern matching algorithm Starting point: the detection of street-names Prefix trees are used for fast text matching for street-names An address-block candidate is constructed by detecting: street names and number postal codes municipal names We will use OpenStreetMap database for global detection 3.2 Postal address detection Street names Street numbers City names Telephone numbers

42. AddressDetection(words) i=0 while i < count(words) set street, number, postcode, city as empty if word[i] is streetName i++ street = words[i] for j = i to i+5 if words[j] is number number = words[j] break for k = j+1 to j+5 if word[k] is postcode postcode = words[k] j = k break for k = j+1 to j+5 if words[k] is city city = words[k] i = k+1 break if street is not empty AND number is not empty AND city is not empty candidate = (street, number, postcode, city) 3.2 Postal address detection Joen Pizza Special Y-tunnus: 2129577-6 Käyntiosoite: Koskikatu 17 80100 JOENSUU Puhelin: 013-220246 Virallinen toimiala: Kahvila-ravintolat streetName number postcode city

43. Prefix Trees Invented by Friedkin (1960) The prefix tree (or trie) is a fast ordered tree data structure used for retrieval Root is associated with an empty string All the descendants of a node have a common prefix of the string associated with that node Some nodes can have associated values (usually they mark the end of a word)

44. Street-name prefix trees Our solution is to detect street-names using prefix trees constructed from the gazetteer A street-name prefix tree is build for each municipality used in the search The user’s location and his area of interested are known, therefore prefix- trees can be limited to municipalities Prefix Tree StatisticsFinlandSingapore Maximum tree depth3414 Average tree depth12.77.4 Average tree width105167 Average number of nodes per tree 23382335 Total size (MB)74.40.18

45. 3.3 Retrieve extra information - Title detection (or company detection) is a Named Entity Recognition problem Usually, the text before the address holds relevant information There are other methods to investigate such as using classifiers or using web page structure Joen Pizza Special Y-tunnus: 2129577-6 Käyntiosoite: Koskikatu 17 80100 JOENSUU Postiosoite Koskikatu 17 80100 JOENSUU Puhelin: 013-220246 Virallinen toimiala: Kahvila-ravintolat address words before the address

46. 4. Ranking Geocoder Web page retrieval Data mining Result ranking User lat, lon keywords web pages result list ranked result list Main criterion: distance from the user’s location Future idea: relevance to user’s profile and history

47. Future ideas recap – Use freely available geographical sources for extending the prototype to other regions – Use geographical scope of a web page to improve address detection and disambiguation – Use the structure of the HTML page and DOM tree semantic analysis for better data extraction – Gather and tag a testing dataset for better evaluation of the algorithms