1. Modeling change from large-scale high-dimensional spatio-temporal array data
Meng Lu and Edzer Pebesma
Institute for Geoinformatics, University of Muenster, Germany
Contact:
Meng Lu
Introduction
Study case 2:
Land cover change detection with historical MODIS satellite data
Research Questions
How to model spatio-temporal change?
How to reduce dimensions spatially and temporally, or thematically?
How to analyze array data?
How can we perform complex spatio-temporal statistical computations on multi-dimensional arrays
How can we combine data sets of different dimensionality or different resolutions into arrays?
The massive data that comes from Earth observation satellites and other sensors provide significant information for modeling global change. At the same time, the high dimensionality and large size of the data has brought about challenges in data acquisition, management, effective querying and processing. In addition, the output of earth system modeling tends to be data intensive and needs methodologies for storing, validating, analyzing and visualization, e.g. as maps. An important proportion of earth system observations and simulated data can be represented as multi-dimensional array data, which has received increasing attention in big data management and spatio-temporal analysis. Array based data management and analysis brings opportunities in modeling change with large-scale high-dimensional data.
3-Dimensional array data (EVI2 )
Study site:
Juara, Mato Grosso, Brazil
Juara
Longitude
0.02
0.23
0.26
0.21 …
0.1
0.36
0.28
0.25
0.4
0.22
0.27
0.3
0.24
0.09
0.13
0.44
0.19 .
brazil
Mato Grosso
Latitude
time
MODIS tiles
Step one: time series analysis with BFAST
Step three: bring the analyses back to SciDB
Step two: joined spatial temporal analyses
Make BFAST Parameters ‘Spatial’
* t
Examples of Array Data
make the parameters of the seasonal effect spatial, i.e. represent them as spatially correlated random fields, and examine the residuals from that to find break points. (spatial correlation model such as CAR, SAR )
The 13 years, 8-day, 250m resolution MODIS satellite images that cover this area has been organized as a dense three dimensional spatio-temporal array and loaded into SciDB for change modeling. The spatial coordinates and temporal information can be retrieved from the array index.
Spatial dependence
Multidimensional Map Algebra
Parameters of trend
1-D: Time series D: Satellite images D: Image time series D: Sediment/nutrient in flow D: Hyper-spectral remote sensing time series data
Spatio-Temporal Kriging
Implementation of Multidimensional Map Algebra (MMA)
Study case 1
Rainfall analyses with fixed sensor data
Dimension reduction (i.e. PCA (Principle Component Analysis)) and time series analyses for rainfall data in two watersheds– small semiarid watershed (Walnut Gulch Experimental Watershed) with densely distributed rainfall gauges and a much larger watershed (Minnesota River Basin) with sparsely distributed rainfall gauges
Meaningfully scaling the time dimension to make the spatio-temporal dimensions statistically equal. Spatial temporal sampling
reflectance
Methodology:
Parameters of seasonality
Two study cases were developed to investigate the potential of array data in spatio-temporal change modeling. The two cases focus on different types of array data: fixed-sensor data with regular (e.g. daily rainfall data) or irregular (e.g. rainfall event data) time series; and satellite images. Multi-dimensional array-based database management and analytics systems such as Rasdaman, SciDB, and R will be applied to these cases. In the later stage, study cases might be developed to integrate data coming from different sensors for more detailed information both in space and in time.
Source:
Mennis 2010
For spatio-temporal change modeling:
How to borrow the strength from the spatio-temporal neighbors when spatio-temporal correlation exists; what spatio-temporal model to use?
For implement spatio-temporal models with multidimensional array data and realize the interaction between R and SciDB or Rasdaman:
How to extend conventional map algebra to multidimensional (intelligent) map algebra? What data type to use?
2-Dimensional array data
What is the temporal variability of rainfall within the whole watershed ?
What is the spatial variability of the long- term rainfall?
What is the trend, seasonality, and cyclical pattern within the time period
1967
1968
1969 … 1
245
286
256 2
223
271
268 3
234
264
253 ….
Gage ID
Time
Minnesota River Basin
Walnut Gulch Experimental Watershed
Validation with Landsat 5 data and DETER deforestation monitoring system
Data: summer average rainfall amount
Period: 1894 to 2013
Number of gauges selected : 11
Data: summer average rainfall amount
Period:1967 to 1999
Number of gauges selected : 83
Tools:
BFAST residuals
2006
2005
2003
2004
VS.
Temporal change in spatial variability:
Animation of PC2 loadings in four periods gage as variable, time as record
The spatial variability of the long- term rainfall:
Spatial distribution of PC1 loadings (top), PC2 loadings (middle) and PC3 loadings (bottom); gage as variable, time as record
Spatial correlations in BFAST residuals as well as in regression coefficients, both in the periodic component as in the trend components, could be found, which provide rationale for joined spatio-temporal analysis
Database management system +
The changes that detected by BFAST (dots) do not match well with the changes reported with DETER (polygon) *
*with time buffer
Data analytics system
Each of the circle represent the loadings of PC for each gage. The size of the circle indicates the magnitude of the PC loadings (how much the variable contributes to the variance). The blue and black indicated negative and positive of PC loadings.