1. 1 Design and Implementation of EarthScope Data Portal Chaitan Baru, Kai Lin San Diego Supercomputer Center

2. 2 The Goal of EarthScope Data Portal Seismic data located in IRIS GPS data located in UNAVCO Drilling data located in SAFOD Goal: –Develop a unified web interface for searching and downloading EarthScope Data with the common denominators Data Classes: –Seismic, GPS, Strain, Drilling logs, Tilt, Physical sample, Environmental, MT Spatial Coverage Temporal Coverage Networks and stations

3. 3 Architecture of EarthScope Data Portal IRISUNAVCOSAFOD Web Service EarthScope Data Portal AuthenticationLoggingCaching database

4. 4 Web Services for EarthScope Data getStations –Get all stations for specified data classes, time range, bounding box and networks. listData –Get all datasets for specified data classes, time range, bounding box, networks and stations. packageData –Pack selected datasets into a downloadable package and return a package ID. checkPackage –Check the status of a package and return the progress or a URL for downloading http://schemas.earthscope.org/portal/0.7/esdp.wsdl

5. 5 Map Interface Use Google Maps Challenge: more than 5000 stations Our Solutions: –Non-cluster mode: for high-end computers –Cluster mode: for low-end computers The browser sends the current spatial extent and the map size to the server The server put adjacent points into a cluster and sends back to the browser to show on the map Possible Alternative Solutions: –Other web based map servers –Google Earth Plug-in –Google Maps + WMS

6. 6 Search Interface Web Forms for Common Denominators: –Data Classes –Spatial Coverage –Temporal Coverage –Networks and Stations Challenge: memorize the last search parameters Solution: –Search parameters are saved as cookies at the browser side

7. 7 Data Search Response A Data Class (e.g. Seismic, GPS) is a high level category of data. In many cases, refined subclasses are needed (e.g. Raw GPS data, GPS data product based on an algorithm). Up to each data center to organize data search result into a hierarchy that is best to present the datasets. The portal combines possible three hierarchies from each data center to a single hierarchy for end users.

8. 8 Data Cart Similar to Shopping Cart –Persistent, i.e., cross sessions –Users can put selected datasets into the Data Cart. A typical use case of the Data Cart –Search data –Save some datasets into the Data Cart –Repeat the steps above –“Checkout” the datasets in the Data Cart

9. 9 User Tracking Use username to track the user if the user is logged in Use cookie to track the user if the user is not logged in –For each guest user, save a unique user-id as a cookie to identify the user –Every dataset the user selected in the Data Cart is saved in the portal database associated with the user-id – A guest user can view his/her Data Cart and requested data packages same as login users if he/she uses the same browser The logging shows that most users prefer guest user mode

10. 10 Packaging User can request to download all datasets in Data Cart as a package. The size limit of a package is 2GB. The portal sends a request to each data center to pack its own data as a zipped file. Each data center sends back a package ID for tracking the status of the package. Each data center needs time to finish packing data. The portal periodically queries the status of each unfinished package until getting a download URL. The portal downloads the three packages from three data centers and repack them into a single package and let the user download. The final downloadable package expires in 2 weeks.

11. 11 Question?