Customizing ArcGIS Online Data Basin Project Overview Web Mapping for Conservation Organizations Workshop Redlands, California March 8th – 9th, 2010

Agenda Project Goals Project Overview Technical Overview Challenges Outcome Q&A Web Mapping for Conservation Workshop

Data Basin Project Goals for ESRI Support the needs of the conservation community by providing infrastructure to store and disseminate spatial data and maps Gather requirements and provide feedback to ESRI product development while implementing and testing ArcGIS 10 and ArcGIS Online 2.0. Support successful implementation of ArcGIS in a cloud computing environment Conservation needs – in addition, tailoring a system that is geared toward both GIS and non-GIS experts. Web Mapping for Conservation Workshop

ESRI Product Development Teams ESRI Professional Services Team Team Collaboration ESRI Product Development Teams ESRI Professional Services Team CBI Development Team Communication is key. Teams need to have confidence in each other’s abilities and be able to share mutual trust. ESRI Prof services = liaison between CBI development and ESRI product development teams. CBI requirements and feedback relayed by ESRI Prof Services to the ESRI product development teams Product updates from ESRI software development teams relayed by ESRI professional services to CBI Web Mapping for Conservation Workshop

Data Basin Project Lifecycle Planning & Strategy Requirements Analysis Design Development Deployment Operations & Maintenance Typical project life cycle used in Prof. Services development projects. Details on each stage and unique characteristics of each for this particular project. Web Mapping for Conservation Workshop

Planning & Strategy Core ESRI product considerations Requirements Analysis Design Development Deployment Operations & Maintenance Core ESRI product considerations Alignment with ESRI product development release schedule Alignment with CBI Data Basin release schedule Web Mapping for Conservation Workshop

Requirements Analysis Planning & Strategy Requirements Analysis Design Development Deployment Operations & Maintenance Joint review of pre- defined Data Basin requirements Communicating Data Basin requirements to ESRI product teams Gap analysis Web Mapping for Conservation Workshop

Design Planning & Strategy Requirements Analysis Design Development Deployment Operations & Maintenance Collaboratively designed specifications for both Data Basin website and GIS Web services UI design independently conducted by CBI Web Mapping for Conservation Workshop

Development Planning & Strategy Requirements Analysis Design Development Deployment Operations & Maintenance Amazon Cloud development environment (same as ArcGIS Online) ESRI – GIS Web services CBI – development of front end application and non-GIS components & services Web Mapping for Conservation Workshop

Deployment Deployment requirements review Planning & Strategy Requirements Analysis Design Development Deployment Operations & Maintenance Deployment requirements review Performance & scalability testing Amazon Cloud environment managed by ESRI Managed Services Deployment – performance & scalability results will assist ESRI with understanding how well ArcGIS performs on the cloud Web Mapping for Conservation Workshop

Operations & Maintenance Planning & Strategy Requirements Analysis Design Development Deployment Operations & Maintenance Technology Transfer Workshop Public beta launch - May 1, 2010 Full Data Basin release target – July 2010 Data Basin data and application hosting provided by ESRI Managed Services Tech transfer workshop – review ESRI-developed GIS services and perform knowledge transfer to CBI development team so they can maintain the services moving forward. Operation & Maintenance – tech transfer allows for CBI to take over management of all GIS functionality (with ESRI’s support as needed) Web Mapping for Conservation Workshop

Project Schedule ESRI level of effort for overall project = ~1200 hours Development timeline reflects both CBI and ESRI development activities A little over 1.5 years to fully release Data Basin Web Mapping for Conservation Workshop

Overview of Business Requirements Easily find and access high-quality conservation-related data (including essential metadata) Create and share compelling online maps to support conservation work Contribute and publish data and share it publicly or with specified groups of users Allow users to contribute information about themselves and the work they do to foster connections between people Support development and promotion of conservation-related applications Ensure secure access to restricted information Provide full control over branding and design to CBI Web Mapping for Conservation Workshop

Overview of Functional Requirements User and group management Workspace management Searching and browsing Web mapping including base maps Limited GIS analysis Web site content management Web Mapping for Conservation Workshop

Overview of Operational Requirements Build with scalability in mind – start small and evolve over time Initial release with approximately 800 – 1000 datasets Ability to grow number of datasets over time Handle simultaneous access by large number of users Provide reasonable level of availability Limit operational cost and hosting cost Web Mapping for Conservation Workshop

Design Considerations and Options ArcGIS Online Data Sharing fulfilled 80% of requirements User and group management Workspace management Searching, browsing, and Web mapping capabilities Gaps: Exposing Web mapping capabilities for user contributed content requires ArcGIS Server infrastructure Inability to customize the look and feel Lack of management tools for organizations Other missing features (sorting of search results, comments and ratings, spatial search, ability to feature specific content) Web Mapping for Conservation Workshop

ArcGIS Geowarehouse REST API RESTful Web services enabling the sharing features of ArcGIS Online Simple, easy to use model Users Groups Content items Accessible to any registered ArcGIS Online user REST Services Directory and API Documentation are disabled Web Mapping for Conservation Workshop

Implementation Approach Leverage ArcGIS Geowarehouse REST API using a custom developed Web application Provide control over branding and design Identify common requirements and use cases and include them in the ArcGIS Online product development cycle Utilize and extend the capabilities of the ArcGIS software platform to enable Data Basin specific features and to support an accelerated implementation to meet project schedule Employ Amazon Elastic Compute Cloud (Amazon EC2) Web Mapping for Conservation Workshop

Walkthrough: Automatically Publishing Datasets as Map Services Problem statement: In order to use the Web mapping feature of ArcGIS Online users need to publish ArcGIS Server map services This requirement is impractical considering the target audience of Data Basin Solution: Automatically prepare datasets for publishing as map services and publish them on demand Requires extending the capabilities of ArcGIS Server Web Mapping for Conservation Workshop

Walkthrough: Approaches to Extending ArcGIS Server Create Web services that leverage GIS business logic using ArcObjects Easy to develop and deploy Asynchronous and synchronous request execution Can negatively impact performance due to high network overhead Create custom server object extensions More complex to develop and deploy 9.3.x: Manual XML processing for SOAP; REST requires custom Web service 10: Direct REST and SOAP support in SDK Synchronous request execution only Good for performance sensitive applications Create custom geoprocessing functions Easier to develop and deploy SOAP and REST support out of the box by way of geoprocessing service Both synchronous and asynchronous execution Best for long running processes (>5 seconds) Approach chosen for Data Basin Web Mapping for Conservation Workshop

Walkthrough: Import Workflow 1 2 3 User uploads dataset System creates temp. map service User enters metadata Abstract and summary Field descriptions Custom styling Shapefile Archive (.zip) Layer Package (.lpk) ESRI GRID Create map document Add dataset with default styling Publish map service 5 4 Restore map service Archive map service Delete map service Archive map document and store archive on Amazon S3 Create item in ArcGIS Online Download and extract archive from S3 Publish map service Map service ready for use Map service does not consume resource when it is not being used Web Mapping for Conservation Workshop

Demonstration Web Mapping for Conservation Workshop

Solution Architecture Design Considerations Coarse-grained interfaces Asynchronous processing Stateless components Partitioning of data and functionality N-tier architecture The processing requirements of the layers in the application differ. Processing in one layer could absorb sufficient resources to slow the processing in other layers. Layered architectural style Application is complex, and the high-level design demands separation so that teams can focus on different areas of functionality Support multiple clients

Third-party Technology Technology Stack ESRI Technology Third-party Technology ArcGIS 10 ArcGIS Desktop ArcGIS Server for the Java Platform ArcGIS API for JavaScript 1.6 ArcGIS Geowarehouse REST API 1.3 ArcGIS Online Services Java SE Development Kit (JDK) 6 Python 2.6 Dojo 1.4.1 Sun GlassFish Enterprise Server 2.1 MySQL Community Server 5.0.67 Fedora Linux release 8 Windows Server 2008 Web Mapping for Conservation Workshop

Project Challenges Developing with ArcGIS 10 beta versions Management of joint development activities ArcGIS Online integration and alignment with ESRI core development objectives Amazon Cloud deployment of ArcGIS Integration with ESRI global account system Developing strategies for scaling ArcGIS Server Web Mapping for Conservation Workshop

Project Outcomes Requirements sharing with ArcGIS Online Comments and ratings Sorting search results (by comments, ratings, etc.) Increase user & community focus Testing support for ArcGIS 10 Holistic testing of ArcGIS in an end to end scenario Defect reporting and suggestions for workarounds Testing support for ArcGIS on the Cloud ArcGIS performance & scalability testing on the cloud Amazon deployment experience Requirements sharing – these are just some examples of the requirements that were shared with ArcGIS Online that were built into the product Testing on the Cloud – note that Marwa will go into detail. Web Mapping for Conservation Workshop

Questions Web Mapping for Conservation Workshop