1. Concurrent Web Map Cache Server: A “Web 2.0 Meets SOA” Case Study
Zao Liu, Marlon Pierce, Sunghoon Ko, Geoffrey Fox
Community Grids Laboratory
Indiana University
&
Neil Devadasan
Polis Center
Indiana University Purdue University Indianapolis
It’s a program which could be considered as a web map service based on several existing GIS server.

2. GIS Map Servers as a Web 2.0 Case Study
There are several different products for creating on-line maps and allowing interaction with Geographical Information System (GIS) data bases.
ESRI, Autodesk, Open Geospatial Consortium
These follow a classic user driven request/response style model.
Google Maps (released in 2005).
Highly interactive AJAX style clients replaced stodgy user-driven request/response.
See Workshop for more information.
More importantly, any one could use the JavaScript API to make really sophisticated applications.

3. Building a Hybrid System
Google Maps provide a highly interactive user interface and capabilities (geolocations, directions)
But GIS services have much more detailed local information.
Indiana has orthophotography with much higher zoom levels than Google maps.
Local county servers have many interesting map layers not in Google
Parcels/property lines, school district lines
And these tie into feature services with interesting data like pinpoint addresses, tax assessments, etc.
So obviously it makes sense to adopt the Google approach but enhance it with local data.
Ultimately we hope to ties this into representations of scientific data generated on the Grid.

4. Marion County Map Server (ESRI ArcIMS) Hamilton County Map Server
Google Maps Server
Marion County Map Server
(ESRI ArcIMS)
Hamilton County Map Server
(AutoDesk)
Cass County Map Server
(OGC Web Map Server)
Must provide adapters for each Map Server type .
Cache Server
Tile Server
Adapter
The tile server fulfills Google map calls with cached tiles at the requested bounding box that fill the bounding box.
Cache Server requests map tiles at all zoom levels with all layers. These are converted to uniform projection, indexed, and stored. Overlapping images are combined.
Browser client fetches image tiles for the bounding box using Google Map API.
Browser +
Google Map API 4

5. Building a Cache Server
Reverse engineering Map Server requests.

6. Federating GIS Servers Around Indiana
Indiana has 92 counties
Approximately 15 have public GIS map servers.
Examples
ESRI ArcIMS and ArcMap Server
Marion, Vanderburgh, Hancock, Kosciusco, Huntington, Tippecanoe
Autodesk MapGuide
Hamilton, Hendricks, Monroe, Wayne
WTH Mapserver™ Web Mapping Application (OGC Minnesota Map Server)
Fulton, Cass, Daviess, City of Huntingburg
Also there are state-wide GIS servers
Orthophotography from Indiana University
Indiana Geological Survey
These are not normally interoperable.

7. Map Requests for ArcIMS and ArcMap
Map image requests for ESRI ArcIMS and ArcMap are based on ArcXML
ESRI sends SOAP-like XML over HTTP. Note the generated image is left on the server. Client has to retrieve it in a separate step. Server cleans up images every 10 minutes or so (a configurable parameter).

8. Map Requests for other type of Servers
Map requests for other type of servers are using HTTP GET method.
Trick is to figure out the correct format for the name/value pairs.
Requesting format of MapGuide and WMS is almost the same
Map image is directly returned in the HTTP response (GIF, JPG, etc)
Request for AutoDesk MapGuide Server
Request for WTH Web Map Server

9. County Boundaries
To take advantage of highly accurate local data for use statewide, a variety of technical issues must be overcome such as:
Projecting the information to a single coordinate system
Standardizing symbols
Retrieving individual Layers .

10. Caching for Performance
County Server
Performance is constrained by the performance of the individual servers.
Cache Server
We have implemented two methods to achieve this.
One is an agent server which will wait for a user’s connection, then check the user’s operation to see how many counties’ service it requires, then send requests to all those county servers, wait for their responses and then combine the maps to return to the user.
The response time in this schema is quite long, since the users have to wait enough time for remote county servers to generate maps, plus the time for our agent server to combine the map, then transfer it back.
Besides, in this method, if one county server doesn’t work， our agent server will not response to client neither.
Obviously this is not suitable for AJAX style applications. We need to pre-fetch and store as tiles. 10

11. Building a Tiling Server
Reverse engineering Google’s map server.

12. Tiling Strategy
Google Maps works by delivering map tiles that fill a bounding box.
Google Map API 2.0 lets you point at your own tile server.
We use this to serve up our own map data together with Google maps.
To do this, all tiles should be saved as the same bounding box as Google map tiles.
Tiles must have same size, projection, and coordinate values as underlying Google base maps.
Each tile uses tile ID and zoom level as its name (ie, tile ), so no database is needed to find the tile.
Our tile naming convention based on Google’s lat/lon to transverse Mercator projection.
tileX.Y.Zoom, but must first convert lat and lon to rectangular coordinates.
Naming convention discussed on next slide. See
Using zoom level as the first-level directory and layer as the second-level directory to store tiles.

13. Converting bounding box to Google tile values
Google uses an x,y coordinate system combined with a zoom value to specify the tiles to retrieve from the server. These coordinates are calculated using an algorithm which can be found in GoogleMapki. See:
Find the bounding box of Indiana which covers all the state.
Convert the latitude value and longitude value of the bounding box into the Google map tile coordinate values.
Identify the tile coordinate values in the bounding box.
Convert the tile ID into the latitude and longitude values. Use Indiana Geology Survey’s service to match county to tile. Store in database.
We can pre-fetch tiles by sending request to the county server that these tiles belong to and fetch the tile back.
Steps for fetching image tiles from county map servers that match Google map tiles.
Example Tile:

14. Naming Tiles Example Tile 1 (36,47) Tile 4 (37,47) Tile 2 (36,48)
Convert A, B lat/lon values to Google map tile values at a given zoom level. Value for A is (36,47), B is (37,49)
Calculating how many tiles there are in this bounding box and identify each tile’s value. In this figure, there are ( )*( ) = 6 tiles (our choice).
For each tile in the bounding box, we can convert it’s tile coordinate values into lat/lon values.
Using each tile lat/lon values to construct requests for IGS boundary services. IGS Services will tell which counties are at least partially in this tile. The site for IGS services is:
Save the tile-county mapping in our database. For a given tile name, we can look up the county.
Tile 1 (36,47)
Tile 4 (37,47)
Tile 2 (36,48)
Tile 5 (37,48)
Tile 3 (36,49)
Tile 6 (37,49)
Bounding box of Indiana at zoom level 10
B(-84.6, 37.1)

15. Combine Google map with county parcel data
Map servers typically contain base maps and optional layers.
Parcel boundaries, roads, and township boundaries are layers.
We cache each layer separately.
Layers and base maps are combined dynamically using Java Advanced Image libraries. 15

16. Matching Projections: EPSG4326 to Mercator
County map over-layer from IGS is in EPSG4326 projection. Must convert to Mercator to match Google. 16

17. Combine tiles at County Boundaries
Marion County
Hancock County
The communication between two MapFacher threads occurs on the boundary cases.
As the graph in this slide, while a MapFecher is working on its county’s image retrieving, it might get a map like this. While XmlFetcher request for maps, it might send request for the same coordinates to several county servers if the map covers more than one county. And the image generated by each remote server would seemed like this. So that while the mapfecher is working, it will still fetch this map and then try to break it into tiles, but as we can see, there will be many useless tiles contains nothing in it. In addition, in order to be able to combine to a map across boundary, all the mapfetcher need to store all those tiles even they have the same coordinates.
While our program is running, we found that in many map fetcher’s local directory, it is frequently that many small tiles are empty only few of them contains useful info.
County boundary tiles need to be combined to one tile by use Java Advanced Image Library

18. Next Steps Caching more regions in Indiana and elsewhere.
If county uses ESRI or OGC map server, current agent plugins can be used.
We would like to do California next.
Use to represent outputs of scientific applications.
Contour plots, vector maps, and other types of layers for displaying results of geophysical applications.
Dynamic (“real time” layers) to display streaming data from instruments and applications.
Find a way to keep current with county servers, especially when the county server change layer id.
Recent Monroe county example
The tiling services should support multiple server styles
URLs for REST/AJAX style clients
WSDL and SOAP for formal Web Services
Support OGC and ESRI clients.
Improve collaborative clients

19. Observations and Conclusions
Web 2.0 approaches are very compatible with SOA in general...
Although the details are important.
But we have to do a better job making services.
We are very good at making complicated interfaces to simple services.
Programmable Web lists 350+ simple public APIs to complicated services.
Science Gateways will change dramatically.
We have been burdened down by security issues that Google et al ignore.
Portals tend to be dominated by server-side, Enterprise standards, while mash-ups favor thicker browser clients and looser standards
Judge a service by popularity rather than number of pages in the specification.
And it’s the data...we need to find better ways to use Grids like the TeraGrid to populate data services.
Computations will always require expertise.
Grid software is useful for computing experts.
But not everyone needs this. We need to think of better ways for archiving and delivering computational results.

20. More Information mpierce@cs.indiana.edu See demo:
Collaborative version:
Need a) Flash, and b) a friend to also try.
Buggy still, so you have to login at the same time.

21. Comparison of state and county data
10 foot contours (1990) 1 foot contours (2006)
Missing local roads Local roads (2006)
No parcels Parcels (2006)
No point addresses Point addresses (2006)
Jurisdictional boundaries (2001) Jurisdictional boundaries (2006)

22. Basic Problem: Data Federation
Integrated GIS systems have obvious benefits but inevitably systems are developed by various state and local government agencies.
Bottom up rather than top down
This tends to give excellent local information but it breaks down at the county boundary.

23. Considerations We assume heterogeneity in GIS map and feature servers.
Must find a way to federate existing services
We must reconcile ESRI, OGC, Google Map, and other technical approaches.
Make a clean distinction between clients and services
Must try to take advantage of Google, ESRI, etc rather than compete.
We must have good performance and interactivity.
Servers must respond quickly--launching queries to 20 different map servers is very inefficient.
Clients should have simplicity and interactivity of Google Maps and similar AJAX style applications.

24. Backup Slides

25. Developing issues Integrating GIS map servers is not trivial
Different county map servers may use different technologies and web services.
Interoperability of Geospatial Referencing: Different coordinate systems and projections are used by the different county web services.
Semantic Interoperability: Different attribute names for layers are used by the different county web services.
Our solution: create a virtual map server to act as an agent server
Translates map requests from generic format to the format expected by the specific map server.
Provides a common language and programming interface for constructing clients
Projecting the information to a single coordinate system
Standardizing symbol
The agent server by itself will work but performance is not good
Must wait for slowest server to respond
Failure prone: a county server may not respond at all
Adds additional overhead for combining images

26. Caching Server
The agent server runs offline to harvest map images from county map servers.
Images are stored as tiles.
Tiles at county boundaries may be combined for greater storage and performance efficiency.
Clients connect to the cache server instead of the agent server.
The cache server constructs the requested image from pre-fetched tiles.
Inspired by Google Maps approach
Enable more interactive clients
Image construction may be parallelized/multi-threaded for greater performance.
Potentially takes advantage of new multi-core server architectures from Sun, Intel, and AMD.

27. Two Phase Approach: Caching and Tiling
Federation through caching:
WMS and WFS resources are queried and results are stored on the cache servers.
WMS images are stored as tiles.
These can be assembled into new images on demand (c. f. Google Maps).
Projections and styling can be reconciled.
We can store multiple layers this way.
We build adapters that can work with ESRI and OGC products; tailor to specific counties.
Tiling:
Client programs obtain images directly from our tile server.
That is, don’t go back to the original WMS for every request.
Similar approaches can be used to mediate WFS requests.
The tile server can re-cache and tile on demand if tile sections are missing.
Google Map Clients can work with tiling server. 27

28. Cache Server architecture
IGS boundary- checking service
Tile-County Mapping Process
County Web Map Services
County Map Fetcher thread
County Web Map Services
County Map Fetcher thread
Database
County Web Map Services
County Map Fetcher thread
County Boundary Fetcher thread which need to use JAI library to combine tiles from different counties.
County Web Map Services
County Web Map Services
1 - 28 28

29. Building Indiana Map client Using Google Map API
Google Map API V2 enables us to add custom map directly on Google Map .
See
To utilize Google map API to build Indiana map client, there are several steps:
Register for using Google map API
Create a collection of copyrights.
Give those copyright collections to all tile layers that we want to add on Google Map.
Pass those tile layer(s) off to create a map type.
Once a map type has been created, it can be added to our map instance in our JavaScript code. 29

30. Example code for building custom map on Google Map
Create a GCopyright
var copyright = new GCopyright(1, new GLatLngBounds(new GLatLng(-90, -180), new GLatLng(90, 180)), 0, Geology Survey");
Add the GCopyright to a GCopyrightCollection
var copyrightCollection = new GCopyrightCollection('Chart'); copyrightCollection.addCopyright(copyright);
Create a GTileLayer array
var tilelayers = [new GTileLayer(copyrightCollection , 3, 11)]; tilelayers[0].getTileUrl = CustomGetTileUrl;
function CustomGetTileUrl(a,b) {
var z = 17 - b;
var f = "/maps/?x="+a.x+"&y="+a.y+"&zoom="+z;
return f; }
Create a GMapType
var custommap = new GMapType(tilelayers, new GMercatorProjection(12), "Chart", {errorMessage:"No data available"});
Add the custom map type to the map
map.addMapType(custommap);
Once the maptype added to the map container, this type layer could be controlled as Google map. 30

31. Collaborative Indiana Map
Flex data service from Adobe which is the basic technology used to support collaboration of Indiana map clients.
Enables innovative applications to be delivered in the browser in a reliable and scalable manner.
Enables data to automatically be pushed to the client application without polling.
Enables a client application to concurrently share data with other clients or other servers. This model enables new application concepts like “co-browsing” and synchronous collaboration.
The Flex module adds the following collaboration features to Google Maps:
Map sharing: Maps are kept in sync (in real time) between users involved in a collaboration session.
Videoconferencing (Webcam sharing and VOIP): You can share your Webcam and microphone to add video and audio to your collaboration session.
White-board: Collaborating users can draw on the map. For example you could draw potential directions, etc. The users’ whiteboards are kept in sync in real time.
Chat-board: Collaborating users can chat on the chat-board in real time.
Cursor sharing: When you move your mouse, other users see the movements of your mouse and what you are pointing at.
Using flex API to communicate with Ajax and Javascript to implement rich web applications. 31

32. Flex runtime architecture
To enable collaboration, clients need flash player first installed.
To the clients who subscribe to the same channel which configured in the flex data services, the events invoked in one client could broadcast to other clients to enable collaboration.
Clients can make direct calls to Java objects as well as subscribe to real-time data feeds, send messages to other clients, and integrate with existing Java Message Service (JMS) messaging systems. 32

33. Storage of caching entire state
Currently storing 15 counties at 13 zoom levels for 13 layers. It is takes ~250 GB.
Takes about TB to store the entire state to zoom level 13 this way.
There are tiles for zoom levels 0-13,  tiles for 0-14 levels (nearly 12 TB).
There are ~10 layers for each scale
Aerial photo layer tiles take 25~30 KB
Other layers (parcels, roads) are much smaller: 30~36 KB for all remaining 9 layers per tile
So we need almost 60KB * tiles to store all map data
Layers from Google (Hybrids, Street, Google Satellite) don’t need to
be cached.
This is large but possible.
We can easily spread our caching server over multiple hosts to store even higher magnification scales.
Efficient tiling storage can save disk space. 33

34. Current Progress Supports ESRI and OGC servers
Now 17 counties is being cached. (Marion, Monroe are fully cached for 13 zoom levels)
7 layers has been proved that they can be easily cached.
Aerial photo layer, street , interstate layer, parcel, parcel ID, county boundary, school).
3 more layers can be easily shown in client without caching. (Google Map, Google Satellite, Hybrids).
Querying parcel information across boundary. ( MARION-HANCOCK boundary)
Support Geocode querying.
Higher resolution than Google Satellite.
Google Map-like interaction.
Performance and Reliability.
Cache Server still work even the county server doesn’t work.
Much faster response to the client. 34

35. Tradeoffs of Caching Cached images must be store somewhere.
More zoom levels, much more disk space is needed.
For 12 levels, GB.
For 13 levels, TB.
For 14 levels, about 12 TB. (It may be not necessary to cache this zoom level for all counties. We can cache this level for the requirement of some place.
Difficulty of map re-projection.
Latency of keeping update with county servers.
Inconsistencies in available layers. 35