1. C LOUD C LEARING R OUND R OBIN CLAIRE BULGIN, CHRIS MERCHANT UNIVERSITY OF READING

2. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 2 S ESSION I NTRODUCTION Meeting Schedule 1.GlobTemperature – Cloud Clearing Round Robin (CCRR) Objectives and Design. 2.Aerosol CCI – Round Robin Objectives 3.Cloud CCI – Benefits from participating in CCRR. 4.Discussion

3. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 3 G LOB T EMPERATURE CCRR C ONTEXT Cloud clearing is a fundamental pre-processing step in land surface temperature (LST) estimation from satellite data. Cloud detection algorithms are being developed by a number of groups using a variety of approaches. A cloud clearing round robin encourages competition between groups to develop the best cloud detection algorithm over land. This can then be applied in the generation of a LST climate data record from the Along Track Scanning Radiometers.

4. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 4 CCRR P ARTICIPATION Participation from members of other projects also needing to develop or apply cloud detection techniques. Eg. Aerosol CCI, Cloud CCI, GlobSnow, GlobAlbedo. Identification of potential participants as part of the User Requirements Survey for LST. Raising awareness of the CCRR at meetings and conferences. Literature review of current cloud detection techniques using visible and infrared data.

5. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 5 CCR R ECRUITMENT AND C OMMUNICATION Invitations to participate in the cloud clearing round robin will be communicated amongst the scientific community in the following ways: GlobTemperature Website Events Presentations Science Networks (eg. EarthTemp) Direct invitation to groups actively involved in developing cloud detection over land.

6. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 6 C LOUD C LEARING R OUND R OBIN P ROTOCOL This will define how the Cloud Clearing Round Robin will be run, and will cover the following elements. Documents for participants. Schedule for participation. Definition of the protocol for algorithm selection. Set out the rules of participation. Define the metrics for algorithm selection. Define the content of the Round Robin Data Package distributed to all participants.

7. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 7 CCRR O BJECTIVE P ROCEDURE Development/Training DatasetTest DatasetSelection Dataset Submissions Apply Assess Apply Submit Comparisons 1 st Data Release2 nd Data Release Cloud Mask SuppliedBlind Data

8. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 8 CCRR S CENE S ELECTION AND C LASSIFICATION Cloud masks will be determined by expert inspection of nadir and forward view ATSR imagery using multi-channel and stereo methods. 3 Datasets will be created: Development/training Test Selection Datasets will include a range of land surface types (Desert, vegetated, snow, ice), atmospheric conditions (high/low water vapor, clear and aerosol rich) and surface temperatures.

9. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 9 CCRR D ATA P ROVISION For each scene, we will provide all of the information required for a full Bayesian cloud detection scheme, from which algorithm developers can use some or all of the data. The fields provided will include: ATSR Reflectance Channels ATSR Brightness Temperatures NWP Surface Temperature NWP Total Column Water Vapor Manual Cloud Mask (with the exception of the selection dataset). Land/sea mask Simulated reflectance and brightness temperatures (RTTOV 11.1). Data quality information Land cover information

10. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 10 P ERFORMANCE M ETRICS Code for calculating the performance metrics (which will be used in the final algorithm selection exercise) will be provided to all participants to use in their developmental work, as part of the Round Robin Data Package. Potential performance metrics include: Percentage of perfect classification. Hit Rate False Alarm Rate True Skill Score LST Impacts Performance under clear conditions vs. high aerosol loading. Performance over cold/reflective surfaces. Eg. Snow, ice, desert.

11. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 11 R OUND R OBIN D ATA P ACKAGE This will be distributed to all participants and will include: A description of the round robin protocol. The round robin datasets (distributed according to the schedule). A description of the datasets. Software for calculating performance metrics (identical to that to be employed in the algorithm selection process). Distribution will be done via a zipped tar file from a FTP site.

12. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 12 D EVELOPMENT AND S UBMISSION S UPPORT Development Following the distribution of the Round Robin Data Package, participants will be given support in accessing and using the data. Responses to questions will documented on a webpage or in an ‘FAQ’ document (where not already covered in the data package documentation) to facilitate troubleshooting. Submission The test and selection datasets will be disseminated together in accordance with the timing specified in the schedule. The time period between this dissemination and the submission data will be short to prevent further tuning and development of algorithms. Re-submission will only be allowed in the event of formatting problems or corrupt data in order to maintain objectivity in the algorithm selection process.

13. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 13 A NOTE ON INTERNAL ALGORITHM SUBMISSION … Algorithms submitted to the round robin from internal partners will be subject to the same protocol as external submissions. Development will only be possible using the training dataset. Development, test and selection datasets will be disseminated at the same time to all participants and internal parties will not have access to the selection data cloud mask to ensure objectivity.

14. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 14 CCRR – A LGORITHM S ELECTION Full analysis of algorithm performance including all scenes, algorithms and metrics will be extensive and therefore presented online. The selected algorithm will be implemented with the LST climate data record production chain. Performance of the selected algorithm in large-scale application will be reviewed in case there are problematic circumstances outside the range of situations covered by the round robin exercise.

15. © ESA – GlobTemperature Consortium CCRR Meeting, 16 th June 2014, Harwell DUE GlobTemperature 15 D ISCUSSION P OINTS Scene selection – what range of scenes are required to generate results applicable to all projects? Does there need to be an expert aerosol mask as well as a cloud mask? Are more metrics required that are aerosol/cloud specific? How will the workload be spread between the GlobTemperature and Aerosol CCI projects? Are other auxiliary data fields required for aerosol classification? Are there other avenues of communication/invitation required for the Aerosol and Cloud CCI’s?