1. WMP Technical Working Group – 14 Feb 2012 …and the Watershed Management Program!

2. WMP Technical Working Group – 14 Feb 2012 Last meeting we discussed the difficulties handling large numbers of ArcHydro-generated catchments, specifically including: 1- processing time to get the “micro-catchments”, 2- time to review “micro-catchments”, and 3- difficulty constructing the HydroNetwork for “micro-catchments”, and 4- problems with Preferential Flow Path and catchment grouping. Recent Developments: 1- Some empirical data on processing times and some surprising findings, 2- Ideas to cut down on review time, 3- A fix for the HydroNetwork construction, and 4- no real progress on catchment grouping (yet.) Well… three out of four isn’t bad!

3. WMP Technical Working Group – 14 Feb 2012 1 – Some Empirical Data Experimental Conditions: Dell T3500 – Win7-64 SP1; Intel Xeon Dual Core (3.2GHz); 4GB RAM ArcGIS 10: Builds 2414, 2800 (SP1), or 3200 (SP2) ArcHydro 2: Builds 2.1.0.16, 2.1.0.19, 2.1.0.22, 2.1.0.27) Notes: Some combinations of ArcGIS 10 service packs and ArcHydro 2 demonstrated incompatibilities. ArcGIS 3200 and ArcHydro 2.1.0.16 and ArcHydro 2.1.0.19 produced unexpected errors. This was corrected with 2.1.0.22. We find the most stable combinations to be ArcHydro 2.1.0.22 and.27 with ArcGIS 2414 or 3200. We have not tested ArcGIS SP3 Time trials included processing up to HydroNetwork (Drainage Connectivity Characterization tools)

4. WMP Technical Working Group – 14 Feb 2012 Brooker Creek (Hillsborough County portion) Basin Area ~23.5 sq. miles DEM Size = 56,000,000 cells SWF13023 Sink Fill Area (in US Acres)4356 sq ftNoFIll Number of Catchments/Sinks1,703418,542 Sink PreScreening0:04:070:04:00 Sink Evaluation0:05:550:48:45 Fill Sinks0:00:340:00:55 Flow Direction with Sinks0:04:200:12:30 Flow Accumulation0:02:130:01:16 CatchmentGridDelineation0:00:53 Catchment Polygon Processing0:00:210:13:34 Drainage Point Processing0:00:280:16:22 Drainage Boundary Definition0:00:481:24:18 Sub-Total Processing Time0:19:393:02:33 Drainage Connectivity (HydroNet)0:03:267:08:47 Preferential Flow0:03:110:00:00 TOTAL PROCESSING TIME0:26:1610:11:20 ArcHydro statistics: BoundaryDrainage lines – 5,248 vs. 1,231,134 Sink Points – 1703 vs. 418,542 Hydro Edges – 10,059 vs. 2,453,996 HydroJunctions – 6,934 vs. 1,600,863

5. WMP Technical Working Group – 14 Feb 2012 Just a real quick calculation… 1 Day = 24 hours = 1440 minutes = 86,400 seconds So…to review (just look at) each catchment in Brooker Creek for just 1 second each…. 418,542 catchments (seconds) / 86,400 seconds/day = 4.85 DAYS That is non-stop, doing ABSOLUTELY NOTHING else!

6. WMP Technical Working Group – 14 Feb 2012 Pemberton Creek – Hillsborough County Pemberton Creek Basin Area ~65 sq. miles DEM Size = 156,000,000 ArcGIS Build2414 SP0 ArcHydro Build2.1.0.19 ComputerSWF13325 Sink Fill Area (in US Acres)0.1 acre fillNo Fill Number of Catchments/Sinks1158965,076 Sink PreScreening0:13:510:13:06 Sink Evaluation0:18:381:50:21 Fill Sinks0:02:490:02:50 Flow Direction with Sinks0:10:248:06:46 Flow Accumulation0:10:300:03:17 CatchmentGridDelineation0:03:500:03:05 Catchment Polygon Processing0:00:230:30:25 Drainage Point Processing0:00:590:34:32 Drainage Boundary Definition0:00:413:15:02 Sub-Total Processing Time1:02:0514:39:24 Drainage Connectivity (HydroNet)0:03:5517:03:00 Preferential Flow TOTAL PROCESSING TIME1:06:0031:42:24

7. WMP Technical Working Group – 14 Feb 2012 LinesTime North12286631:21:04 North22907351:41:32 North32116851:15:01 Central42546761:32:49 Central52362541:24:37 Central62875341:45:03 Central71667141:02:16 Central8985800:37:25 Central91599140:58:55 Central102083521:15:44 Central111544060:58:30 Central121469000:56:46 South131532960:57:53 South142008801:15:25 Total17:03:00 Pemberton Creek – ArcHydro HydroNetwork “Chunks” The Drainage Connectivity Characterization Tool works on a selected set of Catchments (= ArcHydro “Chunk”)

8. WMP Technical Working Group – 14 Feb 2012 Pemberton Creek Catchments

9. WMP Technical Working Group – 14 Feb 2012 Raleigh-Rogers-Starvation Lake Watershed Basin Area ~ 69.5 sq. miles DEM Size = 156,000,000 cells SWF13023 Sink Fill Area (in US Acres)1 acre0.5 acre0.25 acre4356 Sq. Ft.No Fill Number of Catchments/Sinks1,9162,0973,78580,147865,418 Sink PreScreening0:12:060:11:54 0:11:530:11:59 Sink Evaluation0:05:470:05:570:06:470:14:481:38:35 Fill Sinks0:02:030:02:400:02:370:03:160:03:44 Flow Direction with Sinks0:12:040:11:10 0:09:0213:01:00 Flow Accumulation0:06:390:06:100:06:010:04:400:03:45 CatchmentGridDelineation0:02:300:02:290:02:230:02:010:02:42 Catchment Polygon Processing0:00:34 0:00:440:04:070:28:05 Drainage Point Processing0:00:55 0:00:580:03:270:37:51 Drainage Boundary Definition0:01:110:01:150:01:420:17:213:44:39 Drainage Connectivity (HydroNet)0:05:250:05:280:08:011:33:1116:01:12 Preferential Flow0:00:00 Total Processing Time0:49:140:48:320:52:172:43:4635:53:32 Raleigh-Rogers-Starvation Lake Watershed

10. WMP Technical Working Group – 14 Feb 2012 Pinellas-Anclote Watershed Watershed Size (sq. Miles) = 63.3 DEM size (number of Cells) = 180,325,425 Processing MachineSWF13322 SWF13023 ArcGIS Build/Service Pack2800 SP0 2414 SP0 ArcHydro Build2.1.0.16 2.1.0.19 Sink Fill Area (in US Acres)Half acre FillQuarter Acre Fill4356 SqFt625 Sq Ft Number of Catchments/Sinks1,9993,3121,6592,332 Sink PreScreening0:17:560:17:500:16:540:15:38 Sink Evaluation0:09:240:09:560:18:201:33:18 Fill Sinks0:04:270:03:280:03:390:03:48 Flow Direction with Sinks0:14:170:13:360:17:070:16:01 Flow Accumulation0:09:520:09:330:12:440:10:24 CatchmentGridDelineation0:05:490:05:040:06:070:05:25 Catchment Polygon Processing0:00:340:00:420:00:310:00:37 Drainage Point Processing0:00:590:01:010:01:070:01:53 Drainage Boundary Definition0:01:100:01:330:01:010:01:15 Drainage Connectivity (HydroNet)0:06:070:08:220:05:310:06:54 Preferential Flow0:00:00 Total Processing Time1:10:351:11:051:23:012:35:13

11. WMP Technical Working Group – 14 Feb 2012 Pinellas-Anclote Watershed Watershed Statistics (625 sq. ft. fill): 2,332 Sinks/Catchments 6,971 Drainage Boundary Lines 9,281 HydroJunctions 13,495 HydroEdges

12. WMP Technical Working Group – 14 Feb 2012 Pinellas-Anclote Watershed Anclote Watershed AncloteBasin HydroJunction HydroEdge Catchment 1:30,000 – 625 sq. ft. fill

13. WMP Technical Working Group – 14 Feb 2012 Pinellas-Anclote Watershed Anclote Watershed AncloteBasin HydroJunction HydroEdge Catchment 1:30,000 – No Fill

14. WMP Technical Working Group – 14 Feb 2012 Pinellas-Anclote Watershed 1:1200 No Fill Watershed Statistics (No Fill) 1,146,540 Sinks/Catchments 3,371,256 Drainage Boundary Lines 3,733,211 HydroJunctions 5,156,619 HydroEdges ArcGIS Build/Service Pack2414 SP0ChunkLinesTime ArcHydro Build2.1.0.19 North14302882:34;59 Sink Fill Area (in US Acres)No FillNorth23333081:59:40 Number of Catchments/Sinks1,146,540North31476050:55:26 North41221120:45:18 Sink PreScreening0:16:59Central53418142:08:15 Sink Evaluation2:11:28Central62309421:28:11 Fill Sinks0:04:00Central72400751:28:59 Flow Direction with Sinks14:37:55Central82275901:25:44 Flow Accumulation0:03:47Central91508660:57:18 CatchmentGridDelineation0:03:58South101405680:53:31 Catchment Polygon Processing0:36:22South112133081:20:04 Drainage Point Processing0:44:59South122819581:45:03 Drainage Boundary Definition3:59:51South135108823:10:05 Drainage Connectivity (HydroNet)18:17:34 Preferential Flow0:00:00 Total Processing Time40:56:53 That is the equivalent of 17.7 days of Catchment evaluation!

15. WMP Technical Working Group – 14 Feb 2012 Conclusions: 1- Some fill is necessary to produce a “reasonable” number of ArcHydro catchments for human evaluation. Results indicate that between 0.1 acre-fill (4360 sq. ft.) and 5x5 DEM cell (625 sq. ft) fill seems appropriate, 2- Filling to one of these thresholds yields ~ 1000 – 3000 ArcHydro Catchments for human evaluation, 3- As the Drainage Connectivity Characterization tool now operates on selected sets, smaller areas of interest (AOIs) can be selected from the DEM and micro-catchments can be generated for those AOIs. Chunking and appropriate workflows can be used to re-integrate AOIs into the HydroNetwork, 4- There are several relationships between the number of elements and processing times. Adding catchments/sinks increases the time for Flow Direction computation but decreases the time for Flow Accumulation computation, 5- The “chunking” workflow permits computing HydroNetworks for LARGE watersheds. Empirical data suggest that the processing takes ~ one hour per 150000 DrainageBoundary lines.

16. WMP Technical Working Group – 14 Feb 2012 Part 2 – ArcGIS Data Reviewer (formerly known as Data ReViewer)

17. WMP Technical Working Group – 14 Feb 2012 Current Status Since 2008, the District has been using a third-party ESRI extension, DataQE, for insuring consistent Data Quality. Advantages: 1- Works in the ArcGIS Environment, 2- User customizable dictionary and rule set, 3- Ruleset can be easily distributed to multiple users, 4- Reasonable learning curve and cost point. Disadvantages: 1- Doesn’t make very good use of the geodatabase, 2- Doesn’t include any topological/geometric checks, 3- Doesn’t track edits and audits made to the database, 4- Doesn’t include any comment functionality (or tracking), 5- Doesn’t record information regarding reviewers and editors, 6- Doesn’t indicate severity of errors.

18. WMP Technical Working Group – 14 Feb 2012 Status under Evaluation for Adoption Prior to ArcGIS 10, ESRI bundled several products into the Production Line Tool Set (PLTS) package, including: Workflow Management Job Tracking Version Management Data Quality Management Although these are still available as a bundled product, in 2011, ESRI began offering the Data Quality Management portion as either a “stand alone” or concurrently licensed extension as: ArcGIS Data Reviewer

19. WMP Technical Working Group – 14 Feb 2012 ArcGIS Data Reviewer Advantages: 1- Tightly integrated into ArcGIS 2- Reviewer Checks include, Database Validation Checks Polygon Checks PolyLine Checks Table Checks Topology Checks Z-Value Checks Spatial Parameter Evaluation Feature on Feature Checks Duplicate Geometry Checks

20. WMP Technical Working Group – 14 Feb 2012 ArcGIS Data Reviewer Advantages continued, 3- Maintains QC calls in feature classes and geodatabase tables, Feature Dataset Reviewer Tables

21. WMP Technical Working Group – 14 Feb 2012 ArcGIS Data Reviewer Advantages continued, 4- Maintains Detailed information in the REVTABLEMAIN, such as: Check types, who made it, severity of error, who repaired it, when/who checked the repair, etc.

22. WMP Technical Working Group – 14 Feb 2012 ArcGIS Data Reviewer Advantages continued, 5- Easy to construct and distribute the rule set. User can select all or a subset of rules to run.

23. WMP Technical Working Group – 14 Feb 2012 ArcGIS Data Reviewer Advantages continued, 6- Facilitates manual review by incorporating a user-defined grid for review, 7- User formatted, flexible report generator, and 8- Insures that corrections are made and QCed! 9- ESRI Maintained and managed software; will always be “in step” with ArcGIS. Disadvantages, 1- Higher cost point than DataQE (a standard ESRI Extension) for a single-use license ($2500 vs $899) 2- Steeper learning curve than DataQE (additional flexibility and more options than DataQE) 3- Requires modifications to current workflows (1 st reviewer, editor, 2 nd reviewer workflow) 4- Additional data tables (and feature classes) are added to GWIS to manage.

24. WMP Technical Working Group – 14 Feb 2012 Moving Forward 1- District Staff have developed rule sets for GWIS including: 2- District Staff are developing and testing workflows for implementation on WMP Projects, 3- District Staff have attended an ESRI Training Session and are developing GWIS-centric Training Materials, 4- Mapping & GIS has deployed ArcGIS Data Reviewer for ArcGIS 10 5- Mapping & GIS will phase out support of DataQE, starting in January 2012 and ending in December 2012. Mapping & GIS will not support DataQE after January 2013.

25. WMP Technical Working Group – 14 Feb 2012 QUESTIONS AND/OR COMMENTS ???