1 Alberta Agriculture and Food (AF) Surface Meteorological Stations and Data Quality Control Procedures

2 Presentation Overview Existing and proposed (AF) network Data QA/QC Parameter list Quality states QA/QC checks Data filling Conclusions

3 Meteorological Station Expansion 67 N-R-T scalable station platforms ►all season ppt (GEONOR) ►temperature ► h umidity ► GOES platform ► 2M wind speed ► Campbell Cr10x-2m loggers Additional sensors can be added later Data will be freely accessible and sensors can be added by any one with dollars, with the caveat that all data would be public domain. Currently 44 are installed and operational 23 more will be operational by May 1, 2008

4 AF Stations: (N = 113) Common Elements ►All season ppt (GEONOR) ►Temperature ► H umidity ►Wind speed 2 m ►GOES platform ► Campbell Cr10x-2m loggers 36 Drought Net Stations (AGDM) Incoming short-wave solar radiation (26) Net solar radiation (3) Wind speed and direction at 10 m (36) Soil moisture and temperature at 5, 20, 50, 100 cm (30) 10 IMCIN Stations Incoming short-wave solar radiation (10) Wind speed and direction at 10 m (10) 67 Agriculture Climate Monitoring Stations (AGCM) Wind direction at 2m (15) Incoming Short-wave solar radiation(15)

5

6 Existing and proposed stations in Alberta’s Near-Real-Time Network AGDM (AF) ACGM (AF) IMCIN (AF) Other (AENV AES SRD) 20 km buffer

7 A QA/QC and Data-Filling Decision Support System for Near Real-Time Climate Data Providing computer-assisted quality assurance, quality control and data filling

8 Parameter List (Hourly) Temperature Humidity Solar Radiation Wind Speed Wind Direction Precipitation (hourly and 6 hourly) Soil Moisture Soil Temperature

9 Quality States Valid Not needed to be checked by a human Suspect Needs to be checked by a human and validated or filled Invalid Needs to be checked by a human and filled Missing Needs to be checked by a human and filled

10 QA/QC Checks Range within a reasonable range Step maximum allowable change Persistence minimum allowable change Like Sensor similar value to similar sensors Spatial similar value to neighboring stations (parameter dependent)

11 Methodology for Defining QA/QC checks We used the hourly period of record supplied by Environment Canada that contains >25 million records from 250 stations in and around Alberta An adjustable trigger point for the “suspect” occurrences was set at 0.01% (1:10,000) for each test Arbitrary and adjustable (default or station specific) For 200 stations 50 hourly values per day

12 Range Checks Three range checks 1. Valid 2. Suspect 3. Invalid If the data falls within the inner range then it will be marked Valid If it falls in between the outer range and the inner range it will be marked Suspect If data falls outside the outer range it will be marked as Invalid If the data is missing it will be marked Missing and then filled

13 Range Checks: Solar Radiation Invalid Suspect Valid Hourly Solar Radiation (W m -2 )

14 Range Checks: Temperature:

15 Data Filling Temporal filling Spatial filling (IDW) Spatial-temporal filling (IDW+) Manual filling In every parameter’s daily rollup you know how many records were filled so you can judge the validity of the daily value

16 Conclusions Relatively dense high quality and scalable network in the Agricultural area of Alberta We have a state of the art QA/QC process that is both flexible and data driven Reduces man power Capable of generating error logs for maintenance checks

17

18 Persistence Check valid susp. valid Difference of Maximum and Minimum over n steps must be greater than y Persistance Checks

19 Step valid susp. valid Difference of maximum and minimum over n steps must be at most y Step Checks

20 Other Tests Like Sensors Relating wind speed 2M to wind speed 10M Relating occurrence of precipitation to humidity Nearest Neighbors

21 Temporal Filling: for most parameters One value missing either side Simply average of two values adjacent values If more than X consecutive values are missing use spatial interpolation Data Filling 3 for most parameters 6 for Soil Temperature 12 for Soil Moisture Missing or Invalid M M M M Up to X values missing linearly interpolate missing values from valid end points

22 Spatial filling: Inverse Distance Weighting Adjustable parameter dependent radius Max 8 neighbors Rainfall = 70 km radius Other = 120 km radius Else use nearest station if within X radius Else use nearest station and mark as suspect Data Filling

23 Spatial-Temporal Filling: Precipitation Total ppt. at Barnwell using IDW = * * *16.4 Distribution of total ppt. by day at nearest station Estimated ppt at Barnwell Data Filling