Extraction of Multi-scale Outlier Hierarchy From Spatio-temporal Data Stream Jianming Lv

Background Outliers: Different spatial patterns Different temporal pattern according to history logs Event related important patterns for intelligent decision Example: Outliers in air quality, traffic flows, house property transaction

Motivation Target: Features of outliers: Traditional methods: Diverse spatio-temporal scale Different scale of outliers imply different semantic context Hierarchical, overlapped and correlated Traditional methods: Discover the outliers of single spatio-temporal scale Target: Extraction of Multi-scale Outlier Hierarchy From Spatio-temporal Data Stream

UI Outlier A: +- %.... A evidence: Outlier B: +- %.... evidence: Spatial temproal A Outlier A: +- %.... evidence: Outlier B: +- %.... evidence: Outlier B: +- %.... evidence: B

Architecture Original Data Stream Multiple spatial scale snapshot sequences with Multiple spatial scale and multiple temporal scale

Architecture - Spatial scale Model outliers as residuals Temporal scale - Prediction model Model outliers as residuals Run on distributed spark + hadoop platform Prediction model based on LSTM Incremental update Fast correlation analysis of outliers in multiple spatial scale and temporal scale

Related work [1]Deepak P. Anomaly Detection for Data with Spatial Attributes[M]// Unsupervised Learning Algorithms. Springer International Publishing, 2016. [2]Zhang L, Zhu Z, Jeffay K, et al. Multi-Resolution Anomaly Detection for the internet[C]// INFOCOM Workshops. IEEE, 2008:1-6. [3]Wu M C, Chen K C. Outlier Detection in Large-Scale Sensor Network Data Using Shrinkage Estimators[C]// IEEE Global Communications Conference. IEEE, 2015:1-6. [4]Li J, Shen J, Chen A. Multi-scale analysis for automatic weather station data[C]// International Conference on Remote Sensing, Environment and Transportation Engineering. IEEE, 2011:4177-4180. [5]Beigi M, Chang S F, Ebadollahi S, et al. Multi-scale temporal segmentation and outlier detection in sensor networks[C]// IEEE International Conference on Multimedia and Expo. IEEE, 2009:306-309. [6]Jun M C, Kuo C C J, Jeong H. Distributed spatio-temporal outlier detection in sensor networks[J]. Proceedings of SPIE - The International Society for Optical Engineering, 2005, 5819:273-284. [7]Wang C, Lin H, Jiang H. Trajectory-based multi-dimensional outlier detection in wireless sensor networks using Hidden Markov Models[J]. Wireless Networks, 2014, 20(8):2409-2418. [8]TIANMIN HU, SAM YUAN SUNG. FINDING OUTLIERS AT MULTIPLE SCALES[J]. International Journal of Information Technology & Decision Making, 2011, 04(2):251-262. [9]Li Y, Gao L. The application of multi-scale wavelet transform in elimination of outliers[C]// International Conference on Environmental Science and Information Application Technology. IEEE, 2010:396-399.

Related work [10]Birant D, Kut A. Spatio-temporal outlier detection in large databases[C]// International Conference on Information Technology Interfaces. IEEE, 2006:179-184. [11]Gupta M, Gao J, Aggarwal C C, et al. Outlier Detection for Temporal Data: A Survey[J]. IEEE Transactions on Knowledge & Data Engineering, 2014, 26(9):2250-2267. [12]Liang J, Parthasarathy S. Robust Contextual Outlier Detection: Where Context Meets Sparsity[J]. 2016. [13]Albanese A, Pal S K, Petrosino A. Rough Sets, Kernel Set, and Spatiotemporal Outlier Detection[J]. Knowledge & Data Engineering IEEE Transactions on, 2014, 26(1):194-207. [14]毛嘉莉, MAO JiaLi, 金澈清,等. 轨迹大数据异常检测:研究进展及系统框架[J]. 软件学报, 2017, 28(1):17-34. [15]O'Leary B, Jr J J R, Xu X, et al. Identification and influence of spatio-temporal outliers in urban air quality measurements ☆[J]. Science of the Total Environment, 2016, 573:55. [16]Rahmani A, Afra S, Zarour O, et al. Graph-based approach for outlier detection in sequential data and its application on stock market and weather data[J]. Knowledge-Based Systems, 2014, 61(1):89-97. [17]Cheng T, Li Z. A Multiscale Approach for Spatio-Temporal Outlier Detection[J]. Transactions in Gis, 2006, 10(2):253–263. [18]Sun Y, Genton M G. Adjusted functional boxplots for spatio‐temporal data visualization and outlier detection[J]. Environmetrics, 2012, 23(1):54-64. [19]Deng M, Shi Y, Gong J Y, et al. A Summary of Spatio-temporal Outlier Detection[J]. Geography and Geo-Information Science, 2016.

Thanks