Lidar Radar Open Software Environment 1.Introduction 2.What is LROSE – A new paradigm 3.The current status 4.Path forward LROSE Presenter: Wen-Chau Lee Contributors: Mike Dixon, Mike Daniels, Charlie Martin, Steve Cohn, Bill Brown Earth Observing Laboratory

NCAR (EOL, MMM) has been developing, maintaining, and upgrading legacy radar software (Solo, REORDER, SPRINT, CEDRIC, ZEBRA, CIDD, etc.) for the community since Evolution of computer architecture and programming languages, and shrinking budgets at NCAR have: made it unfeasible for NCAR to maintain and upgrade this legacy software to meet the community’s needs. required faculty and graduate students to, instead, develop their own software for functionality that cannot be supported by NCAR. made the current software infrastructure difficult to integrate the community’s effort. A new paradigm - The Lidar Radar Open Software Environment (LROSE) white- paper proposal - was developed at EOL, and sent out to 81 members of the community via , to solicit comments and feedback. The the proposal was submitted to NSF in summer NSF approved of the idea in principle, and provided $200K seed funding to EOL to allow initial work to commence in FY13. Background:

We support users in the scientific community, but we are finding it difficult to provide good quality support for aging legacy applications. The scientific community has needs that are not supported by our current software. We have a large code base of software, of varying ages and maintainability. We have inherited data formats that are not optimal for scientific data exchange. What problem are we trying to solve? Spol-Ka at DYNAMO Maldive Islands We need to define a new framework to encourage community in jointly developing future software development – The WRF model

LROSE will develop common software for the LIDAR, RADAR and PROFILER community. It is based on collaborative, open source development. The code would be freely available on the web. The core: developed by NCAR/EOL, largely based on existing code. Algorithms and analysis tools: partly developed and supported by the community. Data would be stored in portable data formats, based on UNIDATA NetCDF. What is LROSE? SPOL Particle ID algorithm DYNAMO 2011/10/16

LROSE-supported LAOF Facilities: H 2 O DIAL WCR S-PolKa 449-Profiler ELDORA CSU-CHILL DOW HSRL HCR

Data exchange formats (EOL/UNIDATA) – Files and data streams in standardized formats – mostly NetCDF using the Climate and Forecasting (CF) conventions, suitable for exporting to models for data assimilation. Core applications (EOL) –Applications that provide the ‘glue’ to hold the system together. Algorithms and analysis tools (Community) – Analysis, research, generating derived products. – Integrate community-developed tools. Displays (EOL/UNIDATA/Community) – For data visualization, and editing as appropriate. LROSE Components: HSRL backscatter, TORERO, 2012/01/12

LROSE high-level data flow:

Collaboration: LROSE is an Open Source project. Cost reduction and efficiency will be achieved through collaboration with other organizations. UNIDATA NOAA/HRD DOE/ARM Universities NCAR/RAL NASA NWS/ROC International: e.g. BALTRAD Europe BOM Australia CSWR CSWR DOW3 ROTATE-2004, Harper, Kansas, May 2004

Enhanced CfRadial radial coordinate format: released updated version of format specification, in response to user community feedback. Radx package: developed software for handling radial coordinate data, and providing algorithms in radial space. Interpolation: Initial development of Radx Radial-to-Cartesian transformation. Legacy software support: SPRINT/REORDER/CEDRIC. Update legacy software: Began work on wrapping CEDRIC in Python interface. Upgrade solo3 radial data viewer: – Released β- version of C++ legacy solo application. – Developed PySolo Python-based solo-like application. – Developed Emerald Matlab-based solo-like application. Progress to mid-2013:

Proposed LROSE Development Timeline Present Year 1 Year 2 Year 3 Web-based data server Wind profiler analysis tools Enhance display tools Data quality control tools Enhance Python wrapper Continue multiple-Doppler radar analysis tools Add/enhance algorithms Add CfRadial to Vchill Lidar algorithms Enhance profiler algorithms Support other display technology – e.g. Google Earth, Vapor, etc. Add/enhance algorithms Documentation Training material Community workshop +$0.8M Define CfRadial format Radx package Legacy software support Wrapping CEDRIC in Python interface Soloiii – C++ β version Python-based Solo Matlab-based Solo Funding Increment Required Enhance CfRadial Enhance Radx package NetCDF interface Code repository Multiple-Doppler analysis tools Solo3 improvements Jazz – data integration tool Release PySolo Add/enhance algorithms Development: EOL will fund work from base at 200K per year Software engineer: 2.5 senior persons x 3 years Science support: 0.3 person x 3 years Management: 0.2 persons x 3 years Steering Committee meeting: every year Long term support: Software engineer: 0.5 senior persons per year Science support: 0.2 person per year Community users workshop: every 2 years External Steering Committee Meeting

The success of this proposal depends on a large body of software that has been previously developed, funded by a variety of sources – under LROSE these will be maintained, supported, integrated and updated. EOL proposes to seek national and/or international partners with appropriate approval/coordination with NSF. Timing of the receipt of resource increments is critical to minimize loss of momentum from phase to phase (keep forward progress of critical staff and partners). Without this extra effort it is likely that access to the majority of the previous work will be lost within the next 2-3 years. Path Forward:

NCAR is supported by the National Science Foundation. Thank you Eldora on the NRL P3

Collaboration LROSE is an Open Source project. Cost reduction and efficiency will be achieved through collaboration with other organizations. UNIDATA: Mohan Ramamurthy Mohan Ramamurthy NOAA/HRD: Frank MarksDOE/ARM: Wanda Ferrell Universities: Bob Houze, Dave Parsons, Sandra Yuter NCAR/RAL: Brant Foote NASA: Walter Petersen, Gerry Heymsfield NWS/ROC: Rich Ice International: e.g. BALTRAD Europe (Daniel Michelson), BOM Australia (Peter May) CSWR: Josh Wurman