1. This material is based upon work supported by the National Science Foundation under Grant No. ANT-0424589. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author (s) and do not necessarily reflect the views of the National Science Foundation. Center for Remote Sensing of Ice Sheets Headquarters, University of Kansas Workshop Details Who: Association of Computer/Information Sciences and Engineering Departments at Minority Institutions (ADMI) faculty/students Where: Elizabeth City State University (ECSU) When: June 7 - July 5 2011 What: A Teach-One-Teach-Many approach to cloud computing Purpose Introduce ADMI to the basics of the emerging Cloud Computing paradigm Understand the computer systems constraints, tradeoffs, and techniques of setting up and using cloud Understand how different algorithms can be implemented and executed on cloud frameworks Evaluating the performance and identifying bottlenecks when mapping applications to the clouds A Cloudy View on Computing workshop and CReSIS Field Data Accessibility Jerome Mitchell 1, Jun Wang 1, Geoffrey Fox 1, Linda Hayden 2 Indiana University 1, Elizabeth City State University 2 Schedule Now I understand Cloud Computing Now I appreciate why Cloud Computing is important Now I really understand Cloud Computing! Parallel Processing Map /Reduce AlgorithmHadoop Twister Programming Model Used by Parallelized by Apache’s implementation CGL’s implementation End of 1 st Week End of 3 rd Week End of 5 th Week TimeIineTimeIine Functional Programming Compute Resources FutureGrid Virtual machines + virtual networking to create sandboxed modules o Virtual “Grid” appliances: self-contained, pre-packaged execution environments o Group VPNs: simple management of virtual clusters by students and educators CReSIS Field Data Accessibility Current CReSIS Data Organization CReSIS’s data products website lists o direct download links for individual files The data are organized by season o Seasons are broken into data segments Data segments are arranged into frames o Associated data for each frame are stored in different file formats  CSV (flight path)  MAT (depth sounder data)  PDFs (image products) File-based data system has no spatial data access support Spatial Data Accessibility Project Two main components: Cloud distribution service and special service for PolarGrid field crew. Data is supported among multiple spatial databases. Google Earth Matlab/GIS GeoServer Spatial Database GeoServer Spatial Database GIS Cloud Service WMS KML Online Data DistributionField Data Access SpatiaLite SQLite Database SpatiaLite SQLite Database Field Data Service Spatial Database Virtual Appliance Spatial Database Virtual Appliance Data Portal Single User Multiple Users (local network) Multiple Users (local network) Virtual Storage Service Virtual Storage Service Cloud GIS Distribution Service Google Earth Example 2009 Antarctica Season Overview of 2009 Flight Paths Data Access for Single Frame SpatiaLite Database o Spatial extension to manages both vector and raster data and supports a rich set of GIS analysis functions through SQL. The data can be directly accessed through GIS software and MATLAB SpatiaLite Database Example 2009 Antarctic flight path data o ~ 4 million entries - originally stored as 828 separate files and imported into one SpatiaLite database file 2009 Antarctica Season Vector Data Visual Crossover Analysis for Quality Control (development project) Flight path data stored as YYYYMMDD_segID_frameID.txt SQLite command to create the segs table: CREATE TABLE segs ( UTCTime Number, Thickness Number, Elevation Number, FrameID VARCHAR(12), Surface Number, Bottom Number, QualityLevel Integer) SELECT AddGeometryColumn ('segs','geometry',4326,'POINT',2) *note: geometry: 2 -> xy, (longitude, latitude), 4326 -> WGS84 coordinate system SpatiaLite: MATLAB Direct Access Mksqlite package: a MEX-DLL to access SQLite databases from MATLAB http://mksqlite.berlios.de/ Add this flag to build.m to enable SQLite to load SpatiaLite as an extension: -DSQLITE_ENABLE_LOAD_EXTENSION=1 Testing in MATLAB: dbid = mksqlite(0,'open', ‘test.sqlite' ) sql = ['SELECT load_extension(''', path_to_spatialite, ''')']; mksqlite(dbid, sql) % load extension mksqlite(dbid, 'SELECT sqlite_version()') mksqlite(dbid, 'SELECT spatialite_version()') mksqlite(dbid, 'SELECT X(geometry) as lon, Y(geometry) as lat from segs where FrameID=2009101601001'); mksqlite(dbid, 'close') References PolarGrid Data Products: https://www.cresis.ku.edu/data SpatiaLite: http://www.gaia-gis.it/spatialite/ Quantum GIS: http://www.qgis.org/