1. Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation Shreyas Shinde Ranjit Iyer

2. Esri UC 2014 | Technical Workshop | Introduction to server administration Command line tools ArcGIS Server Manager Overview of the administrative API Automation Popular scripting workflows - Publishing of services - Setting up permissions - Extracting popular extents - Monitoring services and databases - Scaling and statistics Agenda ArcGIS for Server: Administrative Scripting and Automation

3. Esri UC 2014 | Technical Workshop | Introduction to server administration ArcGIS for Server: Administrative Scripting and Automation ArcGIS Server Installation Media Provision hardware Select a deployment pattern ArcGIS Server Setup Software Authorization Keep it running…

4. Esri UC 2014 | Technical Workshop | The administrator’s toolbox ArcGIS for Server: Administrative Scripting and Automation ArcGIS Server Manager ArcGIS Server Administrator Directory Console tools

5. Esri UC 2014 | Technical Workshop | Located under /arcgis/server/tools A collection of most popular tools Categories - Administrative credentials - For resetting administrative accounts - SSL/certificate management - Backup and restore - Managing cache Lock the tools down using OS file permissions! Command line tools ArcGIS for Server: Administrative Scripting and Automation

6. Esri UC 2014 | Technical Workshop | Command line tools Demo ArcGIS for Server: Administrative Scripting and Automation

7. Esri UC 2014 | Technical Workshop | Browser based interactive application Located at: http://server.domain.com:6080/arcgis/managerhttp://server.domain.com:6080/arcgis/manager Simple, easy-to-use Ability to manage your entire server site - Even though you are accessing Manager on a specific server node Great for common workflows! ArcGIS Server Manager ArcGIS for Server: Administrative Scripting and Automation

8. Esri UC 2014 | Technical Workshop | le Tour de Manager Demo ArcGIS for Server: Administrative Scripting and Automation

9. Esri UC 2014 | Technical Workshop | ArcGIS Server Manager – under the hood ArcGIS for Server: Administrative Scripting and Automation Simple HTTP requests to the administrative API

10. Esri UC 2014 | Technical Workshop | Entire server administration is an API - If ArcGIS Server Manager can use it, so can you! RESTful - GET and POST of JSON data Well organized and well documented Lots of samples Scriptable Designed for automation! Overview of the administrative API ArcGIS for Server: Administrative Scripting and Automation

11. Esri UC 2014 | Technical Workshop | The administrative API ArcGIS for Server: Administrative Scripting and Automation Admin API Services Create and manage your services. Start or stop them as wanted Machines + Clusters Machines + Clusters Add new nodes to grow capacity. Remove them to shrink. Organize machines into clusters. Security Configure GIS tier or web tier authentication. Manage accounts, roles and permissions on services. Configure SSL. Statistics (new at 10.3) Statistics (new at 10.3) Get fine grained statistics on your GIS services. Logs Harvest logs for auditing, understanding usage patterns and troubleshooting. Databases Manage all your database connections through a single interface. System environment Manage system properties, web adaptors, server directories.

12. Esri UC 2014 | Technical Workshop | Common repeated workflows - Plot popular extents for the data Scheduled actions - Restarting services when data is updated - Build map caches during low demand by moving machines to caching cluster Reactive actions - Adding capacity (server nodes) to a cluster when demand increases - Send email notifications when errors are logged Works with all programming/scripting languages that support HTTP! Why automate? ArcGIS for Server: Administrative Scripting and Automation

13. Esri UC 2014 | Technical Workshop | Where should you begin? ArcGIS for Server: Administrative Scripting and Automation

14. Esri UC 2014 | Technical Workshop | Publish Services Demo ArcGIS for Server: Administrative Scripting and Automation

15. Esri UC 2014 | Technical Workshop | Publishing many services Moving services between Dev, Staging, Production environments - Backup/restore is an option but sometimes you just want to republish services Can publish from Desktop and Manager Tedious if you have lots of services Automation is your friend! Motivation ArcGIS for Server: Administrative Scripting and Automation

16. Esri UC 2014 | Technical Workshop | Help system has a script that uses ArcPy ArcPy is an python library that complements the Admin REST API Provides many convenience functions Task : Publish a list of services listed in a file ArcGIS for Server: Administrative Scripting and Automation

17. Esri UC 2014 | Technical Workshop | Works great, just need a few lines of code However, you need ArcPy on the machine Script publishes service definitions sequentially - For every SD - Upload - Publish With many SDs to publish this can be a very slow process Publishing ~100 services took hours Caveats ArcGIS for Server: Administrative Scripting and Automation

18. Esri UC 2014 | Technical Workshop | Run the script from any machine - No ArcPy dependency Exploit concurrency - Parallel uploads - Parallel publishing If your script is amenable to concurrency, exploit it by all means We can do better ArcGIS for Server: Administrative Scripting and Automation

19. Esri UC 2014 | Technical Workshop | Two step process - Upload the SD to the Server - QUICK - Invoke the publishing tool - INTENSIVE Publishing a service with an SD ArcGIS for Server: Administrative Scripting and Automation

20. Esri UC 2014 | Technical Workshop | Script design ArcGIS for Server: Administrative Scripting and Automation Upload Publish SD UpldID SD

21. Esri UC 2014 | Technical Workshop | Comparison results ArcGIS for Server: Administrative Scripting and Automation Speed up of ~3 ArcPy publishing script DIY multi-threaded publishing script

22. Esri UC 2014 | Technical Workshop | Starting a service Demo ArcGIS for Server: Administrative Scripting and Automation

23. Esri UC 2014 | Technical Workshop | Manager Admin Script Options ArcGIS for Server: Administrative Scripting and Automation Start Geometry Service

24. Esri UC 2014 | Technical Workshop | Main function - Drives the logic Internal utility functions - Get Token - Make HTTP requests - Parse JSON responses Anatomy of an Admin script ArcGIS for Server: Administrative Scripting and Automation

25. Esri UC 2014 | Technical Workshop | Apply Permissions Demo ArcGIS for Server: Administrative Scripting and Automation

26. Esri UC 2014 | Technical Workshop | Permissions are the way to secure your service Do it in Manager Tedious for lots of services. Must automate! Task : Apply permissions to services ArcGIS for Server: Administrative Scripting and Automation

27. Esri UC 2014 | Technical Workshop | Show them previously published services in Manager Show them roles Run the script Discuss the script Demo ArcGIS for Server: Administrative Scripting and Automation

28. Esri UC 2014 | Technical Workshop | Health check for services Demo ArcGIS for Server: Administrative Scripting and Automation

29. Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation Get all the folders Using the report API, get all the service instances. Using Geo-services REST API to test availability. Print report

30. Esri UC 2014 | Technical Workshop | Health check for databases Demo ArcGIS for Server: Administrative Scripting and Automation

31. Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation Use findItems to search for all enterprise geodatabases Use validateDataItem to test database connectivity. Print output

32. Esri UC 2014 | Technical Workshop | Popular extents Demo ArcGIS for Server: Administrative Scripting and Automation

33. Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation

34. Esri UC 2014 | Technical Workshop | Elastic scaling Demo ArcGIS for Server: Administrative Scripting and Automation

35. Esri UC 2014 | Technical Workshop | Add machines to reduce latency and increase throughput Dynamically add them when a latency threshold is breached De-allocate machines when traffic is less Task : Dynamically add machines to your site ArcGIS for Server: Administrative Scripting and Automation

36. Esri UC 2014 | Technical Workshop | Start with a single node site Make service requests and watch latency (we’ll use JMeter) Run a script that adds a machine to a cluster Watch the latency drop Demo ArcGIS for Server: Administrative Scripting and Automation

37. Esri UC 2014 | Technical Workshop | Statistics in 10.3 (Sneak peak) ArcGIS for Server: Administrative Scripting and Automation

38. Esri UC 2014 | Technical Workshop | Server is always collecting statistics - Avg response - Number of requests - Number of timeouts -... Report defines the resource, metric and interval - Previous day, last 7 days, last 30 days - Custom time window Server chooses intelligent data point intervals in the reports REST API to harvest all this information Statistics at 10.3 ArcGIS for Server: Administrative Scripting and Automation

39. Esri UC 2014 | Technical Workshop | Python is great!! ArcGIS for Server: Administrative Scripting and Automation

40. Esri UC 2014 | Technical Workshop | Most modern languages (like Python) support Threads Some languages provide concurrency as a language feature - Go - Erlang - Clojure - Scala Suited for concurrent programming Short detour… ArcGIS for Server: Administrative Scripting and Automation

41. Esri UC 2014 | Technical Workshop | Invented at Google About concurrency Lightweight threads (goroutines) Typed thread-safe communication and synchronization (channels) Readable concurrent code Go ArcGIS for Server: Administrative Scripting and Automation

42. Esri UC 2014 | Technical Workshop | Invented at Ericsson Functional language Light weight ‘processes’ Can spawn 100s of thousands of them Concurrency via message passing Actors model Erlang ArcGIS for Server: Administrative Scripting and Automation

43. Esri UC 2014 | Technical Workshop | Python is great!! ArcGIS for Server: Administrative Scripting and Automation Don’t get us in trouble, we don’t want to start a rebellion

44. Esri UC 2014 | Technical Workshop | Thank you… Please fill out the session survey: First Offering ID: 1146 Second Offering ID: 1344 Online – www.esri.com/ucsessionsurveyswww.esri.com/ucsessionsurveys Paper – pick up and put in drop box ArcGIS for Server: Administrative Scripting and Automation

45. Esri UC 2014 | Technical Workshop | Questions? ArcGIS for Server: Administrative Scripting and Automation

46. Esri UC 2014 | Technical Workshop | Resources ArcGIS for Server: Administrative Scripting and Automation GIS Tutorial for Python Scripting Esri Press, 2014 Paperback and e-bookJust released! Offers several hands-on tutorial exercises. Python Scripting for ArcGIS Esri Press, 2013 Paperback and e-bookGood reference text

47. Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation