1. Programming and Automation

2. Agenda Reason for Data automation ModelBuilder Python
Windows Scheduler

3. Strength of GIS is analysis
Automation builds on that strength
Allows the user to control the process but the system runs the process.
Scripts, models and programs

4. Reasons for automation
Easier
Faster
More Accurate
Repeatable

5. In the past ... ESRI AML – Arc Macro Language
SML – Simple Macro Language
Avenue – ArcView 3.2 –
C, C++ - Slowly fading
Visual Basic – Microsoft says bye bye
Perl
Awk

6. Currently ESRI Modelbuilder Python (Arcpy) ArcObjects Java Silverlight
FME

7. ESRI
Simplified process means you don't need a developer to automate tasks
ESRI has 3 way of automating tasks
Model Builder (Any level)
Python Script (Any Level)
ArcObjects (ArcServer required)

8. ArcToolbox

9. ArcToolbox Tools that automate a single task
Buffer
Union
Projections
Easy to use and can be accessed in multiple locations (ArcMap, ArcCatalog, ArcToolbox)
Restricted by license

10. ArcToolbox Contains ESRI tools/scripts 3rd party tools/scripts
Personal tools/scripts

11. Model Builder

12. Modelbuilder Integrated with ArcToolbox Chaining tools together
No programming needed
Can run a single iterator
Workflows can be very simple to very complex

13. Models are created and saved within a custom toolbox.
The toolbox can contain your favourite tools or a set of tools that are used within you model, or just your model

15. Models are created by dragging and dropping tools into the Model Builder
Can contain Multiple processes.
Parameters can be added that require user input
Output workspaces can be controlled

16. Uses Geoprocessing Data Management Cartography/Mapping Buffer Clip
Unions
Data Management
Add xy
Calculate fields
topology
Cartography/Mapping
Representations
Data driven pages

17. Geoprocessing simple
Probably wouldn’t build a model for this

18. Geoprocessing

19. Geoprocessing using a simple iterator (loop)

20. 3 processes in 1 model

21. Administrative

22. Models are not quite fully automated
Require user input/Parameters
Someone has to press the button to run them
Remember to add meta data
Models can be exported to Python

23. Python

24. Python The preferred ArcGIS scripting language
Others can be used
Executes a sequence of steps
Can use conditional logic
Can include numerous iteration (loops)
Can be generated from Model builder or built independently.

25. What is Python
High level language used to automate tasks through programs called scripts
A good beginning language
Logic is fairly simple to follow
Works on a variety of systems (Windows, Linux, Unix)
A version of python is installed when ArcGIS is installed

26. Basic Python X = 5 Y = 3 Print X + Y 8 First = “John” Last = “Masich”
Print “First Last”
JohnMasich

27. Lists >>>suits = ['Spades', 'Clubs', 'Diamonds', 'Hearts']
>>>values = ['Ace', 2, 3, 4, 5, 6, 7, 8, 9, 10, 'Jack', 'Queen', 'King']
>>>print suits[0] Spades
>>>print values[12] King

28. Loops
>>> x = 2 >>> multipliers = [1,2,3,4] >>> for num in multipliers: print x * num

29. Decision structures
>>> x = 3 >>> if x > 2: ... print "Greater than two" Greater than two

30. The Python Window Accessed through ArcGIS
Commands are typed into the window
>>> import arcpy
>>> arcpy.Buffer_analysis(“Nests”, “Buffered_Nests”, “100 meters”, “”,””,”ALL”)
Script can be loaded from an external source

31. Python in ArcMap
arcpy – lets python know you are working with ArcGIS tools and features.
Without it the commands won’t work
Buffer_analysis – refers to the tool that you are running.
Data after that reflects the information that needs to be entered to run the tool
Toolbox – Analysis Tools - Buffer

32. # This script runs the Buffer tool
# This script runs the Buffer tool. The user supplies the input # and output paths, and the buffer distance. import arcpy arcpy.env.overwriteOutput = True try: # Get the input parameters for the Buffer tool inPath = arcpy.GetParameterAsText(0) outPath = arcpy.GetParameterAsText(1) bufferDistance = arcpy.GetParameterAsText(2) # Run the Buffer tool arcpy.Buffer_analysis(inPath, outPath, bufferDistance) # Report a success message arcpy.AddMessage("All done!") except: # Report an error messages arcpy.AddError("Could not complete the buffer") # Report any error messages that the Buffer tool might have generated arcpy.AddMessage(arcpy.GetMessages())

33. Block merge
# # Block merge.py # Created on: :37: # (generated by ArcGIS/ModelBuilder) # Description: # Import arcpy module import arcpy # Set Geoprocessing environments arcpy.env.scratchWorkspace = "C:\\Users\\KFM\\Documents\\ArcGIS\\Default.gdb" arcpy.env.workspace = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb" # Local variables: CFP = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Canfor\\a15384_cfp_blocks.shp" CFP_Reserves = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Canfor\\ExternalReserves_A15384.shp" TKD = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\tkd_consultation_ \\tkd_consultation_blobs_12_18_2012.shp" Mac_Fibre = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\macfibre\\A87345_Development.shp" Conifex = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Conifex\\Conifex_Mackenzie_Blocks_for_KFM_ shp" FTA = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\FTA\\FTN_C_B_PL_polygon.shp" mac_tsa = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Base\\mac_tsa" licencee_blocks = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\licencee_blocks" RESULTS = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\results_blocks" Original_FDU_clip = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\FSP\\Original_FDU_clip" CFP_2 = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Canfor\\a15384_cfp_blocks.shp" Output_Feature_Class = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Canfor\\a15384_cfp_blocks.shp" a15384_cfp_blocks_shp = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Canfor\\a15384_cfp_blocks.shp" CFP_BLOCKS = "C:\\Users\\KFM\\Documents\\ArcGIS\\Default.gdb\\CFP_BLOCKS" Output_Feature_Class__3_ = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\tkd_consultation_ \\tkd_consultation_blobs_12_18_2012.shp" Output_Feature_Class__2_ = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\macfibre\\A87345_Development.shp" FTA_Blocks_shp = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\FTA\\FTA_Blocks.shp" FTA_Blocks_shp__2_ = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\FTA\\FTA_Blocks.shp" Output_Feature_Class__7_ = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\licencee_blocks" Output_Feature_Class__4_ = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Conifex\\Conifex_Mackenzie_Blocks_for_KFM_ shp" results_blocks__2_ = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\results_blocks" Licensee_Blocks_Copy = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\Licensee_Blocks_Copy" licencee_blocks__2_ = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\licencee_blocks" licencee_blocks_clip = "F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\licencee_blocks_clip" # Process: Copy arcpy.Copy_management(licencee_blocks, Licensee_Blocks_Copy, "FeatureClass") # Process: Clip arcpy.Clip_analysis(FTA, mac_tsa, FTA_Blocks_shp, "") # Process: Add Field (4) arcpy.AddField_management(FTA_Blocks_shp, "COMPANY", "TEXT", "", "", "15", "", "NON_NULLABLE", "NON_REQUIRED", "") # Process: Calculate Field (4) arcpy.CalculateField_management(FTA_Blocks_shp__2_, "COMPANY", "\"FTA\"", "VB", "") # Process: Add Field (2) arcpy.AddField_management(Mac_Fibre, "COMPANY", "TEXT", "", "", "15", "", "NON_NULLABLE", "NON_REQUIRED", "") # Process: Calculate Field (3) arcpy.CalculateField_management(Output_Feature_Class__2_, "COMPANY", "\"MFL\"", "VB", "") # Process: Add Field (3) arcpy.AddField_management(TKD, "COMPANY", "TEXT", "", "", "15", "", "NON_NULLABLE", "NON_REQUIRED", "") # Process: Calculate Field (2) arcpy.CalculateField_management(Output_Feature_Class__3_, "COMPANY", "\"TKD\"", "VB", "") # Process: Append arcpy.Append_management("F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Canfor\\ExternalReserves_A15384.shp", CFP, "NO_TEST", "BLOCK_ID \"BLOCK_ID\" true false false 20 Text 0 0 ,First,#,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Canfor\\ExternalReserves_A15384.shp,BLOCK_ID,-1,-1;LICENCE \"LICENCE\" true false false 15 Text 0 0 ,First,#;DIVISION_N \"DIVISION_N\" true false false 254 Text 0 0 ,First,#;AREA_HA \"AREA_HA\" true false false 19 Double 8 18 ,First,#", "") # Process: Add Field arcpy.AddField_management(CFP_2, "COMPANY", "TEXT", "", "", "15", "", "NON_NULLABLE", "NON_REQUIRED", "") # Process: Calculate Field arcpy.CalculateField_management(Output_Feature_Class, "COMPANY", "\"CFP\"", "VB", "") # Process: Dissolve arcpy.Dissolve_management(a15384_cfp_blocks_shp, CFP_BLOCKS, "COMPANY", "", "MULTI_PART", "DISSOLVE_LINES") # Process: Add Field (6) arcpy.AddField_management(RESULTS, "COMPANY", "TEXT", "", "", "15", "", "NULLABLE", "NON_REQUIRED", "") # Process: Calculate Field (6) arcpy.CalculateField_management(results_blocks__2_, "COMPANY", "\"RSLT\"", "VB", "") # Process: Add Field (5) arcpy.AddField_management(Conifex, "COMPANY", "TEXT", "", "", "15", "", "NON_NULLABLE", "NON_REQUIRED", "") # Process: Calculate Field (5) arcpy.CalculateField_management(Output_Feature_Class__4_, "COMPANY", "\"CFF\"", "VB", "") # Process: Delete Features arcpy.DeleteFeatures_management(licencee_blocks) # Process: Append (2) arcpy.Append_management("F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\FTA\\FTA_Blocks.shp;F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\macfibre\\A87345_Development.shp;F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\tkd_consultation_ \\tkd_consultation_blobs_12_18_2012.shp;C:\\Users\\KFM\\Documents\\ArcGIS\\Default.gdb\\CFP_BLOCKS;F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\results_blocks;F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Conifex\\Conifex_Mackenzie_Blocks_for_KFM_ shp", Output_Feature_Class__7_, "NO_TEST", "SHAPE_Length \"SHAPE_Length\" false true true 8 Double 0 0 ,First,#,C:\\Users\\KFM\\Documents\\ArcGIS\\Default.gdb\\CFP_BLOCKS,Shape_Length,-1,-1,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\results_blocks,SHAPE_Length,-1,-1;SHAPE_Area \"SHAPE_Area\" false true true 8 Double 0 0 ,First,#,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\tkd_consultation_ \\tkd_consultation_blobs_12_18_2012.shp,Shape_Area,-1,-1,C:\\Users\\KFM\\Documents\\ArcGIS\\Default.gdb\\CFP_BLOCKS,Shape_Area,-1,-1,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\results_blocks,SHAPE_Area,-1,-1;COMPANY \"COMPANY\" true true false 10 Text 0 0 ,First,#,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\macfibre\\A87345_Development.shp,COMPANY,-1,-1,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\tkd_consultation_ \\tkd_consultation_blobs_12_18_2012.shp,COMPANY,-1,-1,C:\\Users\\KFM\\Documents\\ArcGIS\\Default.gdb\\CFP_BLOCKS,COMPANY,-1,-1,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\ThreeFeathersRecce.gdb\\Licensee\\results_blocks,COMPANY,-1,-1,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\Conifex\\Conifex_Mackenzie_Blocks_for_KFM_ shp,COMPANY,-1,-1,F:\\GIS\\Projects\\2013\\2012_50_ThreeFeathers\\LicenseeData\\FTA\\FTA_Blocks.shp,COMPANY,-1,-1;AREA_HA \"AREA_HA\" true true false 8 Double 0 0 ,First,#", "") # Process: Clip (2) arcpy.Clip_analysis(licencee_blocks__2_, Original_FDU_clip, licencee_blocks_clip, "")

34. PyWin or IDLE
Outside of the ArcMap environment you can work with python in the IDLE program or use an open source such as PYWIN.
Friendlier programming environment.
Create, edit and execute tools

35. Running Scripts through
Python scripts can be run through a basic program such as Windows task scheduler.
Models Can’t be run this way
This allows the user to run memory intensive or longer scripts out side of normal work hours

36. Management Perspective
Cost effective
Controlled output
Less errors
Data is where it should be.
Create more confidence in the results
Doesn’t necessarily mean the are correct!
Process is easily transferred between users