1. Extracting metadata for spatially- aware information retrieval on the internet Pual Clough Presented by Ali Khodaei CS 572

2. Outline Why do we care? Setting Geo-parsing – NER – Improving Geo-parsing Geo-coding – Ambiguity – Improving geo-coding Experiments

3. Why do we care ? Many documents on the web contain geospatial information including addresses, postal codes, hyperlinks and geographic references. This information can be exploited and used to provide spatial awareness to information systems. – transport timetables, routing systems for motorists, map-based web sites and location-based services, etc.

4. Why do we care? A key part of providing such services is the extraction and use of geospatial information. Extracting geospatial references from documents involves two main tasks: – Identifying geographic references : : geo-parsing – Assigning them spatial coordinates : geo-coding

5. Efficiency Criteria Speed : fast execution of geo-parsing and geo-coding programs to allow the processing of large collections within feasible timescales. Reliability : robust processing on typical web data with error-recovery strategies to ensure minimal manual intervention. Flexibility : enable control over the geo-parsing process including the addition of custom gazetteer lists and creation of grammars for context matching. Multilingualism : to be able to process texts written in a variety of languages other than English.

6. Data Sources Several sources of geographic information were available in the SPIRIT project. Each resource contains both place names and spatial information Resources differ in – granularity of place names (e.g. city, state) – Quality of spatial reference (e.g. accuracy) – Type of spatial representation (e.g. point, polygon)

7. Data Sources SABE : Seamless Administrative Boundaries of Europe dataset OS : the Ordnance Survey Gazetteer for the UK TGN : the Getty Thesaurus of Geographic Names

8. SPIRIT Web Collection 94,552,870 web pages approximate size of 1TByte. approximately 9.6 million pages (10.24%) were from European domains; the rest mainly from US sites. in total, 1,759,681 UK, 1,556,585 German, 505,023 French and 270,715 Swiss web pages were extracted for geo-parsing.

9. Geo-parsing

10. Geo-parsing is very related to the more general problem of Named-Entity Recognition (NER) NER is a subtask of information extraction that seeks to locate and classify atomic elements in text into predefined categories (e.g. names of persons, organizations, …)

11. NER Most research on NER systems has been structured as taking an unannotated block of text, such as: Jim bought 300 shares of Acme Corp. in 2006. And producing an annotated block of text, such as: Jim bought 300 shares of Acme Corp. in 2006.

12. NER Most NER algorithms combine gazetteers (lists of known locations, organizations and people) with rules, which capture elements of the surrounding context.

13. Geo-parsing Simple-lookup using gazetteers – Pros Simple Fast Language independent Robust – Cons Fails when entries are not used in geographical sense (e.g. names of people, businesses,..) or When variants of names are used (e.g. historical or vernacular forms) Ambiguity

14. Improving Geo-Parsing Improving geo-parsing (on top of gazetteer) a) Adding context rules b) Filtering out commonly used words c) Using person name lists

15. Improving Geo-Parsing a) Adding context rules – filters candidate locations using context rules to remove stopwords, references to people and organizations, and links to emails/URLs “Mr. Sheffield” is filtered out as a non-geographic reference using the context rule “ -> null” where and are placeholders for entries from the gazetteer lists.

16. Improving Geo-Parsing b) Filtering out commonly used words – All geographical resources used in SPIRIT have entries for places which are more commonly used in a non- geographical sense – “New”, “More”, “Read”, “Guide” and “Old” all appear as entries in the OS gazetteer. – A stopword list for each geographic resource was computed by calculating document frequency from each entry in the gazetteer – The top 1000 place names were manually assessed for any well-known locations (e.g. “Bath” and “Derby” in the UK) and removed from the stopword list.

17. Improving Geo-Parsing c) Using person names lists – In addition to using lists of common words, geo- parsing was extended with lists of person names (both forenames and surnames) to deal with ambiguity resulting from locations being part of a personal name e.g. “John Sheffield”

18. Geo-coding

19. After extracting candidate locations, the second stage involves assigning them spatial co-ordinates (through gazetteer matching).

20. Ambiguity – disambiguating place names with multiple spatial references “Chapeltown” refers to a location in South Yorkshire (UK),Lancashire (UK), Kent County (USA) and Panola County (USA). – degree of ambiguity for each spatial resource is different In OS approximately 8% of place names are ambiguous TGN exhibits the most ambiguity (59%) due to ambiguity between countries as well as within a single country

21. Improving Geo-Coding The simplest (and often most effective) method to resolve referent ambiguity is to assign ambiguous places a default sense, based on – most commonly occurring place – population of the place name – semi-automatic extraction from the Web – the length of hierarchy containing the location – The feature type provided by dataset

22. Improving Geo-Coding Other improvements – Overlap score matching words between place names in the hierarchy of an ambiguous location and those found within n words either side of the name – Preference senses are preferred based on their resources – e.g. SABE -> OS -> TGN senses matching a command-line option specifying the country currently being processed – e.g. preferred country is given a value 1; the rest a value 0

23. Improving Geo-Coding

24. Experiments Geo-Parsing Results correct (C), missing (M), false hits (S), precision (P), recall (R) and F1-measure (F1). From the annotations in the response- set, automatically generated, precision measures the proportion of these matching the manually assigned annotations. From the annotations defined manually, recall measures the proportion of these which are also correctly identified by the geo-parser. The F1 score is a single-valued summary of both precision and recall

25. Experiments contribution of each heuristic to geo-parsing main observation is the effect of not removing the stopwords: the F1 score drops from 0.7148 to 0.5433 (24% decrease) indicating the importance of identifying and removing stopwords.

26. Pros and Cons PROS – Simple and concise – Based on a real system – Real applications CONS – Considers simple/basic cases only – Needs more evaluation results for geo-coding – Not very research-oriented

27. Questions ?