1 Annual Digital Government Research Conference San Diego, CA Project Highlights G.P. Patil May 2006

2 Agency Databases Thematic Databases Other Databases Homeland Security Disaster Management Public Health Ecosystem Health Other Case Studies Statistical Processing: Hotspot Detection, Prioritization, etc. Data Sharing, Interoperable Middleware Standard or De Facto Data Model, Data Format, Data Access Arbitrary Data Model, Data Format, Data Access Application Specific De Facto Data/Information Standard Agency Databases Thematic Databases Other Databases Homeland Security Disaster Management Public Health Ecosystem Health Other Case Studies Statistical Processing: Hotspot Detection, Prioritization, etc. Data Sharing, Interoperable Middleware Standard or De Facto Data Model, Data Format, Data Access Arbitrary Data Model, Data Format, Data Access Application Specific De Facto Data/Information Standard National Applications Biosurveillance Carbon Management Coastal Management Community Infrastructure Crop Surveillance Disaster Management Disease Surveillance Ecosystem Health Environmental Justice Environmental Management Environmental Policy Homeland Security Invasive Species Poverty Policy Public Health Public Health and Environment Robotic Networks Sensor Networks Social Networks Syndromic Surveillance Tsunami Inundation Urban Crime Water Management SurvellanceGeoinformaticsof Hotspot Detection, Prioritization and Early Warning NSF Digital Government Project # PI: G. P. Patil Federal Agency Partnership CDCDOD EPANASA NIHNOAA USFSUSGS Websites: NSF Digital Government surveillance geoinformatics project, federal agency partnership and national applications for digital governance.

3 Geographic and Network Surveillance for Arbitrarily Shaped Hotspots Overview Geospatial Surveillance Upper Level Set Scan Statistic System Spatial-Temporal Surveillance Typology of Space-Time Hotspots Hotspot Prioritization Ranking Without Having to Integrate Multiple Indicators Surveillance Geoinformatics for Hotspot Detection, Prioritization, Early Warning and Sustainable Management Upper Level Set Scan System Definition: A hotspot is that portion of the study region with an elevated risk of an adverse outcome Federal Agency Partnerships CDC DOD EPA NASA NIH NOAA USFS USGS Features of ULS Scan Statistic: Identifies arbitrarily shaped hotspots Applicable to data on a network Confidence sets and hotspot ratings Computationally efficient Generalizes to space-time scan Poset Prioritization System Objective: Prioritize or rank hotspots based on multiple indicator and stakeholder criteria without having to integrate indicators into an index, using Haase diagrams and partially ordered sets. Example: Prioritization of disease clusters with Multiple Indicators National Applications and Case Studies Biosurveillance Carbon Management Costal Management Community Infrastructure Crop Surveillance Disaster Management Disease Surveillance Ecosystem Health Environmental Justice Sensor Networks Robotic Networks Environmental Management Environmental Policy Homeland Security Invasive Species Poverty Policy Public Health Public Health and Environment Syndromic Surveillance Social Networks Stream Networks g Changing Connectivity of ULS as Level Drops G.P. Patil, R. Acharya, W.L. Myers, P. Patankar, Y. Cai, and S.L. Rathbun The Penn State University, University Park, PA R. Modarres George Washington University, Washington, D.C. Example: West Nile Virus First isolated in 1937, this mosquito born disease, indigenous to north Africa, the Middle East and west Asia was first introduced into the United States in Disease Count Quintiles Population Quintiles Disease Rate QuintilesLikelihood Quintiles Comparison of ULS Scan with Circular Scan ULS ScanCircular Scan Confidence set for ULS Hotspot Hotspot Membership Rating Example: Lyme Disease Infections from the bacterium Borelia burgdorfei vectored by ticks from the genus Ixodes. ULS Scan Cylindrical Scan Example: Human-environment indicator values for 16 European countries. There are a total of 3,764,448 admissible linear extensions. The cumulative rank function for Sweden exceeds that of all remaining countries. The crf’s of all countries dominate that of Ireland. The remaining countries cannot be uniquely ordered based on their crf’s. Belgium, Netherlands and United Kingdom have identical crf’s. Admissible linear extensions are comprised of rankings compatible with the rankings of all indicators. Treating each linear extension as a voter, the cumulative rank function is obtained from the frequencies at which each object receives each rank. Disease Rates Comparison of ULS Scan with Cylindrical Scan Year Haase Diagram The crf’s also form a partially ordered set. There are only 182 admissible linear extensions for this poset, yielding the cumulative rank function: One more iteration yields the rankings in the data table. Center for Statistical Ecology and Environmental Statistics

4

5

6 Attractive Features Identifies arbitrarily shaped clusters Identifies arbitrarily shaped clusters Data-adaptive zonation of candidate hotspots Data-adaptive zonation of candidate hotspots Applicable to data on a network Applicable to data on a network Provides both a point estimate as well as a confidence set for the hotspot Provides both a point estimate as well as a confidence set for the hotspot Uses hotspot-membership rating to map hotspot boundary uncertainty Uses hotspot-membership rating to map hotspot boundary uncertainty Computationally efficient Computationally efficient Applicable to both discrete and continuous syndromic responses Applicable to both discrete and continuous syndromic responses Identifies arbitrarily shaped clusters in the spatial-temporal domain Identifies arbitrarily shaped clusters in the spatial-temporal domain Provides a typology of space-time hotspots with discriminatory surveillance potential Provides a typology of space-time hotspots with discriminatory surveillance potential Hotspot Detection Innovation Upper Level Set Scan Statistic

7 A small sample of the circles used

8 Issues

9 Spatial Temporal Surveillance

10 Network Analysis of Biological Integrity in Freshwater Streams

11 Syndromic Crisis-Index Surveillance

12 (left) The overall procedure, leading from admissions records to the crisis index for a hospital. The hotspot detection algorithm is then applied to the crisis index values defined over the hospital network. (right) The -machine procedure for converting an event stream into a parse tree and finally into a probabilistic finite state automaton (PFSA). Syndromic Surveillance

13 Ocean SAmpling MObile Network OSAMON

14 Scalable Wireless Geo-Telemetry with Miniature Smart Sensors Geo-telemetry enabled sensor nodes deployed by a UAV into a wireless ad hoc mesh network: Transmitting data and coordinates to TASS and GIS support systems

15 Upper Level Set (ULS) of Intensity Surface Hotspot zones at level g (Connected Components of upper level set)

16 Changing Connectivity of ULS as Level Drops g

17 ULS Connectivity Tree Schematic intensity “surface” N.B. Intensity surface is cellular (piece-wise constant), with only finitely many levels A, B, C are junction nodes where multiple zones coalesce into a single zone A B C

18 A confidence set of hotspots on the ULS tree. The different connected components correspond to different hotspot loci while the nodes within a connected component correspond to different delineations of that hotspot

19 Genetic Algorithm Scan

20

21

22

23 We also present a prioritization innovation. It lies in the ability for prioritization and ranking of hotspots based on multiple indicator and stakeholder criteria without having to integrate indicators into an index, using Hasse diagrams and partial order sets. This leads us to early warning systems, and also to the selection of investigational areas. Prioritization Innovation Partial Order Set Ranking

24 HUMAN ENVIRONMENT INTERFACE LAND, AIR, WATER INDICATORS RANK COUNTRY LANDAIRWATER 1 Sweden 2 Finland 3 Norway 5 Iceland 13 Austria 22 Switzerland 39 Spain 45 France 47 Germany 51 Portugal 52 Italy 59 Greece 61 Belgium 64 Netherlands 77 Denmark 78 United Kingdom 81 Ireland for land - % of undomesticated land, i.e., total land area-domesticated (permanent crops and pastures, built up areas, roads, etc.) for air - % of renewable energy resources, i.e., hydro, solar, wind, geothermal for water - % of population with access to safe drinking water

25 Hasse Diagram (Western Europe)

26 Figure 17. Hasse diagram of Poset B (left) and a decision tree enumerating all possible linear extensions of the poset (right). Every downward path through the decision tree determines a linear extension. Dashed links in the decision tree are not implied by the partial order and are called jumps. If one tried to trace the linear extension in the original Hasse diagram, a “jump” would be required at each dashed link. Note that there is a pure-jump linear extension (path a, b, c, d, e, f) in which every link is a jump.

27 Figure 18. Histograms of the rank-frequency distributions for Poset B.

28 Cumulative Rank Frequency Operator – 6 An Example of the Procedure 16 The curves are stacked one above the other and the result is a linear ordering of the elements: a > b > c > d > e > f

29 Cumulative Rank Frequency Operator – 7 An example where F must be iterated Original Poset (Hasse Diagram) a f eb c g d h a f e b ad c h g a f e b ad c h g F F 2

30 Cumulative Rank Frequency Operator – 8 An example where F results in ties Original Poset (Hasse Diagram) a cb d a b, c (tied) d F Ties reflect symmetries among incomparable elements in the original Hasse diagram Elements that are comparable in the original Hasse diagram will not become tied after applying F operator

31 Incorporating Judgment Poset Cumulative Rank Frequency Approach Certain of the indicators may be deemed more important than the others Certain of the indicators may be deemed more important than the others Such differential importance can be accommodated by the poset cumulative rank frequency approach Such differential importance can be accommodated by the poset cumulative rank frequency approach Instead of the uniform distribution on the set of linear extensions, we may use an appropriately weighted probability distribution , e.g., Instead of the uniform distribution on the set of linear extensions, we may use an appropriately weighted probability distribution , e.g.,

32 15 European Monitoring Stations Evaluated by 5 Criteria Abb.NUMESTPRBM AUS12121 BEL12122 DEN22022 FIN11211 FRA12222 GER22221 GRE10200 IRE00011 ITA20211 LUX00010 NET11021 POR10201 SPA00110 SWE12010 UNK10212

33 Hasse Diagram for the EEC 15*5 Dataset Abbr.Rank of EEC5 FRA1 GER2 BEL3 UNK4 DEN5 ITA6 AUS7 FIN8 NET9 POR10.5 SWE10.5 SPA12 GRE13 IRE14 LUX15

34 Regions in the Coordinate Space of POSAC Y YpYp 0 P XpXp X (III) Region of points incomparable to P (II) Region of points greater than P (III) Region of points incomparable to P (I) Region of points smaller than P

35 Three profiles in a two-dimensional coordinate space Y ,2,0,2,2 2,2,2,2,1 2,0,2,1,1 X

36 POSAC Plot of Data Matrix 15*5

37 POSAC Output Proportion of profile pairs correctly represented: Label DIM 1 DIM 2 Joint Lateral Fit GER FRA BEL DEN AUS FIN UNK ITA NET POR SWE GRE IRE SPA LUX

38 Atlantic Slope Consortium Case Study: Watershed Indicators Evaluation

39

40 Level3Level2 1 WATERSHEDBIBIFIBINO3 BU FIRBAINVSHA STR MFP SWR_IND EX For estLDIIMP MP AT CORF ORLand-index Back River Cattail Creek Gwynn Falls Saint Mary's A Southeast Creek Upper Patuxent Ahoskie Buffalo Creek Chickahominy Christian Creek Clearfield Creek Conodoguinet A Grindle Creek Little Contentnea Mantua Middle Creek Middle River Pamunkey Repaupo White Deer Creek Wisconisco

41

42 SHA Score Incision Ratio # Stream Stressors Stream-Wetland-Riparian (SWR) Index Buff0-300Basal AreaInvasives# FP-WL Stressors Floodplain-Wetland Condition Conceptual Model of Condition Used for SWR Index

43 Conceptual Model of Condition Used for Land Index MFORCORFOR Land Index IMPLDI Urbanization Fragmentation Forest

44 Analysis of Indicators for Stream Stability Assessment at Bridge Crossings Case Study

45 PROBLEM Bridges fail as a result of the dynamics of the stream channel that they cross.

46

47 INDICATORS The stream channels and bridges can show signs of warning before catastrophic failure of a bridge occurs.

48 Stream Stability Assessment Method Total Score Watershed and Regional Indicators Watershed and Floodplain Flow Habit Channel Pattern Channel Confinement Local Channel Indicators Bed Material Bar Development Obstructions Bank Stability Indicators Bank Soil Texture Bank Slope Angle Bank Protection Bank Cutting Bank Failure Channel Alignment with Bridge Johnson, J. of Hydraulic Eng. 2005

49 49 Bridges with 13 Indicators

50 POSAC plot for the 13-indicator data matrix

51 Concordance Measure

52

53 Concordance Correlations Correlations with Direct Ranking from Concordance Fractional Weights w/ Original Poset Rank: w/ Equal Weights Index: w/ Modeled Weight Index: Correlations with Concordance Based Weighted Differential Poset Ranking w/ Original Poset Rank: w/ Equal Weights Index: w/ Modeled Weight Index: 0.928

54 METEOR Applied to Bridge Data: Hasse Diagram Progression

55

56

57 Rank range run sequence for the 13-indicator data set

58 Maximality level vs. consistency level for the 13-indicator data set

59 End-member Elimination results for the 13- indicator data set

60 Digital Governance and Hotspot GeoInformatics for Monitoring, Etiology, Early Warning, and Sustainable Managment A collection of white papers prepared for the workshop at dg.o2006 The 7 th Annual International Conference on Digital Government Research San Diego, California May 21, 2006

61 Digital Governance and Hotspot GeoInformatics for Monitoring, Etiology, Early Warning, and Management Around the World

62 The five year NSF DGP project has been instrumental to conceptualize hotspot geoinformatics partnership among several interested cross-disciplinary scientists in academia, agencies, and private sector around the world. A declared need is around for statistical geoinformatics and software infrastructure for spatial and spatiotemporal hotspot detection and prioritization. Our efforts are driven by a wide variety of case studies of potential interest to government agencies involving critical society issues, such as public health, ecosystem health, sensor networks, robotic networks, social networks, video mining, homeland security, early warning, and disaster management.

63 1.Siena, Italy (October, 2005) 2.Parma, Italy (March, 2006) 3.Hyderabad, India (December, 2005) 4.San Diego, USA (May, 2006) 5.Okayama, Japan (November, 2006, January, 2006) 6.Bangkok, Thailand (November, 2005) 7.Kuala Lumpar, Malaysia (December, 2005) 8.Jakarta, Indonesia (January, 2006, 2007) 9.Jalgaon, India (December, 2006) 10.Banglor, India (December, 2006) 11.Belo Horizonte, Brazil (April, 2007) 12.Nairobi, Kenya (March, 2007) Course Instructor and Workshop Leader: G.P. Patil Short Course and Case Studies Workshops Around the World

64 [1]Patil, G.P., Geoinformatic Hotspot Systems (GHS) for Detection, Prioritization, and Early Warning. Project hightlights, dg.o2005, Atlanta, Georgia. [2]Patil, G.P. and Taillie, C., Upper level set scan statistic for detecting arbitrarily shaped hotspots. Environmental and Ecological Statistics, 11 (2004), [3]Patil, G.P. and Taillie, C. Multiple indicators, partially ordered sets, and linear extensions: Multi-criterion ranking and prioritization. Environmental and Ecological Statistics 11 (2004), [4]Patil, et al., Upper level set scan statistic system for detecting arbitrarily shaped hotspots for digital governance. Poster presentation, dg.o2005, Atlanta, Georgia. [5]Patil, et al., Geoinformatic surveillance of hotspot detection, prioritization and early warning. Demo, dg.o2005, Atlanta, Georgia [6]Patil, et al., Hotspot geoinformatics for digital governance. Encyclopedia of Digital Government, Ari-Veikko Anttiroiko and Matti Malkia (eds.), (2006, to appear). [7]Patil, et al., Hotspot Detection and Prioritization GeoInformatics for Digital Governance. Digital Government Research Textbook, Larry Brandt, Valerie Gregg, et al. (eds.), (2006, to appear), Springer, New York. [8] Patil, et al., (2006) On PULSE: The Progressive upper level set scan statistic system for geospatial and spatiotemporal hotspot detection. In: The 7 th Annual International Conference on Digital Government Research, San Diego, CA, May 21-24, [9]Patil, et al., (2006) Multicriterian prioritization and differential weights, stepwise aggregration, Hasse diagrams, and Poset cumulative rank frequency operators. In: The 7 th Annual International Conference on Digital Government Research, San Diego, CA, May 21-24, 2006 References

65 (A project website providing project activities, products, publications, and events.) otspot (Article on the workshop program on hotspot geoinformatics.) (Freeware for circular spatial scan program and information.) (Freeware for academia for Hasse program for Windows.) (Website for the Penn State Center for Statistical Ecology and Environmental Statistics—Home base of the digital government research project for hotspot geoinformatics.) Online Resources

66 Digital Governance and Hotspot GeoInformatics Research Programs 1.Parma, Italy 2.Jalgaon, India 3.Bogar, Indonesia 4.Yokayama, Japan 5.Bello Horizonte, Brazil 6.University Park, PA, U.S.A.