We previously proposed a Sub-path Enumeration and Pruning (SEP) approach [8] Sub-path interestingness is measured by Sameness Degree (an algebraic aggregate function of slopes) Compute piecewise slope, flag top and bottom a percentile segments as abrupt units (user given a ). Sameness degree (SD) ranging [0, 1] is a function of piecewise slopes in a sub-path Define a test of the pattern: SD≥θ(given threshold) Enumerate all the intervals in the data and test with the above criterion using SEP. Decompose the statistical function into simple functions (e.g., SUM, COUNT) and pre-compute. Row-wise strategy: For each end unit, examine longer intervals first. Need further pruning. Top-down strategy: For all the intervals, always examine longer ones before its subsets. We further proposed a SEP with Pruning Border (SEPPER) approach that further optimize the enumeration The strategy combines linear (row-wise) and top-down searching strategy Space-time complexity superior to both SEP row-wise and top-down approaches SEPPER’s Time complexity ≤ Min{SEP top-down, SEP row-wise} SEPPER’s Space complexity≤ Min{ SEP top-down, SEP row-wise } The Sahel region in Africa is prone to severe drought due to climate change What is unique about the Sahel? Narrow transitional zone between rainforest and desert (ecotone) Environmental attributes (e.g., vegetation cover) change sharply Vulnerable to climate change Discovering Interesting Sub-paths in Spatiotemporal Datasets {xun, shekhar, {liess, Xun Zhou 1, Shashi Shekhar 1, Stefan Liess 2, Peter K. Snyder 2, Pradeep Mohan 1 1 Department of Computer Science and Engineering, 2 Department of Soil, Water and Climate, University of Minnesota 1. Motivation We developed a data mining approach named SEP to find intervals of abrupt change in eco- climate data. Case studies on real datasets show that the proposed approach can discover major spatial and temporal intervals of abrupt change. Further, we developed the SEPPER approach which improved the computational efficiency over SEP. Experimental evaluation verified the tradeoff between the two previous SEP design decisions and show dominance of the new SEPPER algorithm over them. Support for this research was provided by the following grants: National Science Foundation under Grant No , III-CXT IIS , IGERT DGE , CRI:IAD CNS , USDOD under Grant No. HM , HM , and W9132V- 09-C [1] D. Nikovski and A. Jain. Fast adaptive algorithms for abrupt change detection. Machine learning, 79(3): , [2] G. Narisma, J. Foley, R. Licker, and N. Ramankutty. Abrupt changes in rainfall during the twentieth century. Geophysical Research Letters, 34(6):L06710, [3] A. Dai, P. Lamb, K. Trenberth, M. Hulme, P. Jones, and P. Xie. The recent sahel drought is real. International Journal of Climatology, 24(11): , [4] I. Noble. A model of the responses of ecotones to climate change. Ecological Applications, 3(3): , [5] E. Page. Continuous inspection schemes. Biometrika, 41(1/2): , [6] Tucker, C.J., J.E. Pinzon, M.E. Brown. Global inventory modeling and mapping studies. Global Land Cover Facility, University of Maryland, College Park, Maryland, [7] Joint Institute for the Study of the Atmosphere and Ocean(JISAO). Sahel rainfall index. [8] Xun Zhou et al, Discovering Interesting Sub-paths in Spatiotemporal Datasets: A Summary of Results. In Proc. ACM SIGSPATIAL GIS (GIS’11) pp Chicago, IL, USA, Conclusions 8. Acknowledgements and References 1. Case study on Africa vegetation cover (NDVI) dataset[6] Experiment Setup: Climate model forecast time series (WCRP-CHFP IRI ECHAM4p5-MOM3-DC2fmt2 ATM) Synthetic data sequences with piecewise slope generated using Gaussian distribution Variables: Pattern Length Ratio (PLR): ratio of interesting interval length against data length Data length 6. Computational Speedup The change is persistently abrupt W 1 =[12N, 17N] W2W2 W3W3 4. Approach 5. Case Study: Interesting Sub-path with Abrupt Change 2. Case study on Sahel rainfall index data[7] Vegetation cover in Africa, August 1-15, Major spatial abrupt changes (ecotones) found : the Sahel and the southern boundary of tropical rainforest Other ecotones with abrupt changes in vegetation cover in the world: the Gobi Desert (Asia), Western America, etc Hypothesis: these areas may also be vulnerable to Sahel-like eco-changes. Major temporal abrupt changes found by the proposed approach Decreasing period in late 1960s and mid 1980s[2]. Long decrease ( )[3] found when using larger abruptness percentile parameter. We found long periods of persistently high/low precipitation using a slightly modified interest measure. Upper: Smoothed yearly Sahel rainfall index. Lower: (left): abrupt precipitation change found when using a=0.25 and θ =0.5. (center): abrupt precipitation change found when using a=0.25 and θ =0.3, (right): persistent high/low precipitation periods. Abrupt vegetation cover change in Africa, August 1-15, Vegetation cover map (in NDVI) of Africa (left), abrupt change of vegetation cover in Africa in August 1981 (middle), and global analysis for the same period (right) Source: 1.Sahel rainfall index data, Joint Institute for the Study of the Atmosphere and Ocean (JISAO) Foley et al, Regime Shifts in the Sahara and Sahel: Interactions between Ecological and Climatic Systems in Northern Africa. Ecosystems (2003) 6: 524–539http://jisao.washington.edu/data/sahel/ Vegetation cover of Africa from the GIMMS NDVI dataset [6] Related statistical methods (e.g., CUSUM[1, 5]) only find collections of interesting time-points/spatial locations (e.g., with abrupt changes), rather than long intervals of change. Some other work in climate research [2] are not completely automated. 3. Novelty Abrupt change in a sample data sequence found by CUSUM[5] (left figure, location 6) and our work (right figure, interval 5-11) Location in the data sequence The length of the change intervals vary The interestingness of the sub-path may not exhibit monotonicity (e.g., a long decreasing interval may contain a short increasing part) The data volume can be very large. 2. Challenges Are there other regions that share similar features in the world? Help understand and predict possible severe climate impacts Find spatial intervals of abrupt changes Vegetation cover (in NDVI) along 18.5E longitude Alternative example: a similar pattern can be found in time series Rapid increase/decrease of precipitation/temperature in a few years Help identify events such as droughts from historical data. Precipitation time series in the some region in Africa