Evaluation of RM3 Weather Forecasts Over Western Africa During the 2013 Summer Monsoon Dr. Leonard M. Druyan 1 ; Dr. Matthew B. Fulakeza 1 ; Ruben Worrell 2 ; Lucien Simpfendoerfer 3, and Ari Rubinsztejn 4 1 Team Principal Investigator (GISS), 2 Education Specialist (NYCRI), 3 Undergraduate (NYCRI), 4 High School Student (NYCRI) Sponsors of 2014 NYCRI: National Aeronautics and Space Administration (NASA) NASA’s Goddard Space Flight Center (GSFC) NASA’s Goddard Institute for Space Studies (GISS) New York City Research Initiative (NYCRI) Contributors: Leonard M. Druyan, Ph. D (PI) Matthew B. Fulakeza, Ph.D (PI) Ruben Worrell (Education Specialist) Lucien Simpfendoerfer (Undergraduate) Ari Rubinsztejn (High School Student) Abstract The West African Monsoon (WAM) is a seasonal reversal of winds that brings a season-long period of heavy precipitation to the region. Its arrival indicates the onset of the wet season that Africa’s agricultural economy relies so heavily on. To help the region minimize the effects of climate change on its economy, we must first understand how the WAM will change. Several factors, including changing patterns in sea surface temperatures (SST’s), aerosols, and increasing concentrations of greenhouse gases, may affect the behavior of the intertropical convergence zone (ITCZ), and therefore the variability of the monsoon. The global climate model (Model E2) developed at NASA GISS helps to predict climate changes. However, this model has some deficiencies capturing climatic features, perhaps at least partially due to its lower spatial resolution. The Columbia University/NASA GISS has therefore developed a regional climate model, the Regional Model 3 (RM3), which has better spatial resolution, and can be driven by either reanalysis or Model E2 data, to hopefully help predict these changes. In this study, our goal was to test the RM3’s facility in making daily forecasts when driven by the Global Forecast System (GFS), a global weather model developed by the National Center for Environmental Prediction (NCEP). This was the RM3’s first evaluation while not driven by reanalysis. It is worth mentioning that the RM3 was not developed to produce daily forecasts while being driven by the GFS; instead, it was developed for long-term simulations. We ran the model, gridded at 0.5º, and compared point forecasts for 52 African weather stations with observations made by those stations. Results show that the RM3 underestimated precipitation in the northern Sahel, and overestimated precipitation in the southern Sahel, with the disparities increasing as the rainy season progressed. This implies that the model did not bring the ITCZ far enough north. Overall, precipitation forecasts are slightly overestimated. The RM3 often predicts precipitation when it doesn’t rain, and predicts too little precipitation when it rains especially heavily. The RM3 underestimates maximum temperature forecasts, and overestimates minimum temperature forecasts. Diurnal range forecasts are half as large as observed ranges. Correlations between forecast and observed values for precipitation, maximum temperature, and minimum temperature are highest around Mauritania, around Lake Chad, in the rainforest area along the border between Cameroon and the Central African Republic, and along the northern coast of the Gulf of Guinea. Correlations between forecast and observed maximum and minimum temperatures are also high in the far northern Sahel around Niger. Root mean square errors (RMSEs) for precipitation are higher when average precipitation amounts are higher. Maximum temperature RMSEs decrease from June through early August, and then increase, with average maximum temperatures, while minimum temperature RMSE’s do not show any interseasonal trends. Threat scores are often between 0.4 and 0.6, which shows that precipitation forecasts are encouraging. This evaluation of the RM3’s performance when forced by the GFS demonstrates the RM3’s strengths and weaknesses. We hope that it will hint at how the RM3’s performance can be improved. Map of African Stations Threat Scores Forecast Minus Observed References Druyan L, Mesoscale analyses of West African summer climate: focus on wave disturbances. Climate Dynamics volume (27), p Druyan L,Fulakeza M, Lonergan P, "The impact of vertical resolution regional model simulation of the west African summer monsoon.”, International Journal of Climatology volume (28), p Druyan L, Fulakeza M, "The impact of the Atlantic cold tongue on West African monsoon onset in regional model simulations for ", International Journal of Climatology. Anthes R, Kuo Y, Hsie E, Low-Nam S, Bettge T, "Estimation of skill and uncertainty in regional numerical models.”, Q.J.R meteorol soc. volume (115), p Druyan L, Fulakeza M, Lonergan p, Worrell R,,"Regional Model Nesting within GFS Daily Forecasts Over West Africa.", The Open Atmospheric Science Journal volume (4), p Cook K, "Climate science: The mysteries of Sahel droughts.", Nature Geoscience volume (1), p J. Huang, C. Zhangand, J. M. Prospero,"Large-scale effect of aerosols on precipitation in the West African Monsoon region.", Quarterly Journal of the Royal Meteorological Society Jones, B, "Africa_WorldRegionsNoText". Retrieved July, 2014 Available: National Center for Environmental Protection,, "24 hr Total Precipitation". Retrieved July, 2014 Available: NASA,, "TRMM Online Visualization and Analysis System (TOVAS)". Retrieved July, 2013 Available: Conclusion The ITCZ didn’t move far enough north. Areas south of its actual northernmost position received far less rain than forecast, especially during July and August, and areas near its actual northernmost position received far more rain than forecast. Precipitation forecasts, averaged over the entire region, were slightly too high. The RM3 often predicted rain when it did not rain, and when it rained heavily, the RM3 often didn’t predict enough rain. Forecasts underestimated daily maximum temperatures along a stretch from the West African coast to the northern coast of the Gulf of Guinea, and along the Sahel from the Gulf of Guinea to the eastern edge of the region. Forecasts overestimated daily average temperatures around the CAR, around Mali, and along the Senegal coast. Forecasts overestimated minimum temperatures almost everywhere in the region under study. Average observed diurnal range (TMax – TMin) was twice as large as forecast. Correlation coefficients for precipitation, maximum temperature, and minimum temperature were most frequently significant around Mauritania, around Lake Chad, around the CAR, and along the northern coast of the Gulf of Guinea. TMax and TMin, but not precipitation forecasts were often statistically significant in the Niger/far-northern Sahel region. However, correlation coefficients were significant less often than not. RMSEs for all stations over the entire region on a single day were higher on days when average precipitation for all stations over the entire region was higher: when we expressed daily RMSEs as a percentage of the average precipitation on that day, all temporal trends in RMSEs disappeared. Such RMSEs for daily maximum temperatures showed the same relationship to average maximum temperatures. Minimum temperature RMSEs were relatively steady, and did not change so closely with average minimum temperatures. RMSEs calculated for each station over the entire period were generally very large. RMSEs for precipitation, for example, were often five times the average daily precipitation for that station over the entire period. Threat scores showed that precipitation forecasts are encouraging: for the 0 mm threshold, they were frequently between 0.4 and 0.6, which is considered encouraging for our purposes. Forecast Minus Observed Correlations and Forecast vs. Observed Time Series Root Mean Square Errors Figure 1: Forecast minus observed spatial trends for precipitation, maximum temperature, and minimum temperature. Dark blue = strong negative bias. Lighter blue = slight negative bias. Orange = slight positive bias. Red = strong positive bias. Yellow = varies month to month. Figure 2: Seasonal (JJAS) frequency plots for forecast minus observed values for (a) precipitation, (b) maximum temperature, and (c) minimum temperature. Figure 3: Spatial distribution of the percentage monthly of forecast vs. observed correlation coefficients significant at the 0.10 level in 2013 for (a) precipitation, (b) maximum temperature, and (c) minimum temperature. Brown = 25%-49%, Red = 50%-74%, Yellow = 75%-99%, Green = 100% ab c Figure 6: Frequency distribution of the precipitation threat scores for each station at the 0 mm threshold, for (a) June, (b) July, (c) August, and (d) September. Figure 5: Daily RMSE time series, for (a) precipitation, (b) maximum temperature, and (c) minimum temperature. Time series extends from June through September, Figure 4: Season-long time series. On the left are stations that had high correlation coefficients, to the right are stations that had low correlation coefficients. (a) Precipitation in Parakou, (b) maximum and (c) minimum temperature in Agadez, (d) precipitation, (e) maximum temperature, and (f) minimum temperature in Bamako/Senou. Spatial Comparisons Figure 3: 24 hour precipitation 00Z, 7/22/2014, as forecast by (a) GISS RM3 and (b) GFS. Actual estimates from (c) TRMM. The RM3 captured the southern part of TRMM’s precipitation maximum around Guinea/Liberia/Sierra Leone, but the GFS didn’t. The RM3 showed a maximum around Cameroon that neither the TRMM nor GFS showed. The TRMM had the maximum a little farther west, around Nigeria. The RM3 also had a maximum around the DRC that the GFS showed, but that TRMM didn’t pick up. Both the GFS and RM3 missed precipitation in most regions farther north than 15ºN. a b c