1. World Renewable Energy Forum May 15-17, 2012 Dr. James Hall

2. Earth Observation Satellites -Worldwide coverage -Data access from a single source Regional Ground-Based Networks -Built for specific purposes -Complete coverage requires aggregating many networks Within 75 km ground-based sensors will generally provide more accurate estimates of irradiation. Credit: NASA Credit: ESA

3. Using the 75 km threshold, the continental US could be covered by 450 well- placed sites. Similarly, the EU could be covered with 250 sites.

4. We currently have historical hourly GHI (global horizontal irradiance) data from over 5000 ground sites for the past 5-30 years.

5. Using the 75 km threshold, these 5000 sites provide ground-based GHI data for essentially the entire US.

6. A new system is now capturing real-time GHI and meteorological information from 2800 US and 160 EU ground sites. By Dec 2012 we plan to be ingesting 4800 US and 1000 EU sites in real-time. Observation frequency varies from 5 to 60 minutes. Typical latency in the observations is about 10 minutes.

7. When this real-time system is completed it will provide complete coverage of the US and EU with higher accuracy irradiance data and lower latency than can be obtained from satellite-based observations.

8. Ingest Ingest Timing Controller Queue for Data Capture Processing Convert to Common Format Quality Control Meteorological Data Quality Control Solar Data Solar Forecast Spatial-Temporal Model of Solar Data Seasonal Patterns Short-Term Solar Forecast Data Archive Today’s Data Temporary Folders for Each Network Weather Forecast

9. Ingest Timing Controller Queue for Data Capture Network Folders Network Folders The controller accesses a history of when the observations from each site have been available. At the appropriate time, the controller forms a unique query for the site and places it in the queue. A processing computer executes the queries as efficiently as possible and stores the responses in the appropriate folders. To be considerate of the data provider's bandwidth and server resources, if the data is unavailable after several attempts a log of the event is sent to a separate system for backfilling. Challenge: access thousands of sites from 100+ networks. Observations are asynchronous and may be missing or delayed.

10. Convert to Common Format Quality Control Meteorological Data Quality Control Solar Data Data Archive Temporary Folders Challenge: Data is ingested in over 100 different formats and must be converted and quality controlled in real-time All observations and measurement units are converted to a common format. Typically one or more networks change their format each week. Reprogramming to adapt to these changes is a major part of the effort. Each observation time is converted to universal time. We have found that the stated times on the observations may be in error, especially when networks cross time zones or change to/from daylight savings.

11. Convert to Common Format Quality Control Meteorological Data Quality Control Solar Data Data Archive Temporary Folders Sensors not passing multi-level quality control are excluded from the daily model. Each meteorological parameter is compared to a 30 year historical database for 4000+ locations. Solar QC1 (quality control 1) compares the irradiance to a historical database of observations on the same day of the year for 300+ regions. Solar QC2 compares the irradiance to that of nearby sites at similar elevations. Solar QC3 estimates the calibration and bias of each sensor during clear sky events.

12. Definition: The forecasting of the weather within the next six hours is often referred to as nowcasting. In this time range it is possible to forecast smaller features such as individual showers and thunderstorms with reasonable accuracy. Traditionally, nowcasts have relied on extrapolation and interpretation of the observations. Numerical Weather Prediction models focus on physics. Nowcasts depend on observations that are accurate, high resolution, and real-time. (better data = better nowcasts) Given the current state of NWP models for nowcasting, there is still a place for “human” interpretation of the observations and discovery of seasonal patterns.

13. Today’s Data T+1 T+2 T+3 Time Now T-1 T-2 T-3 Nowcast Historical The 4-D surface is like a movie of irradiance for the past few hours and extending several hours into the future The nowcast extrapolates both the current sky clarity and the system dynamics. This technique merges several forecasting technologies such as persistence, cloud motion vectors, cloud formation & dissipation, etc. At regular intervals the irradiance data is used to generate a spatial- temporal model for the day.

14. On 4/23/2012, 358 sites in California & Nevada passed the quality control filters.

15. This 4D representation shows longitude (X), latitude (Y) and time of day (Z). The size of the balls represent GHI from morning (bottom) to evening (top). Rotating the plot, one can visualize the CA & NV state outlines.

16. Hourly average GHI W/m2

17. This ensemble forecasting model is adaptive and capable of learning based on the historical data presented and the forecast success. Forecasts of Irradiance up to 4 Hours Ahead

18. The physiology of corn and soybean plants has been extensively modeled. When the genetics and local environmental factors are accurately known, it is possible to forecast daily growth and the final yield. Accurate and timely data on local solar radiation has been a missing link. Plant growth & yield Genetics ++ Soils & Farm Practice + Climate +++

19. When the real-time ingest is completed it will offer nearly complete coverage of the US and EU with higher accuracy irradiance data and lower latency than can be obtained with satellite-based observations. Beginning with California, we will be setting up a website with real-time nowcasts of solar irradiance 0 to 4 hours forward and live accuracy analysis of previous forecasts. Beginning in 2013 we will begin forecasting US crop yields for corn and soybeans at the local, state and national levels.