COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC Readiness Status

COSMIC retreat October 2005 UCAR COSMIC Project Office Thanks CDAAC team! Doug Hunt - General architecture + lots more Maggie Sleziak-Sallee - Operator interface Sergey Sokolovskiy - Inversion algorithm Bill Schreiner - Orbits, Clocks, fiducial network Stig Syndergaard - Ionosphere Martin Lohmann - Q/C + Errors Karl Hudnut - Computers/facilities + System support Jim Johnson - Fiducial data, BitGrabber network, Rx tests TaeKwon Wee - 1DVar code Lidia Cucurul - Data assimilation and encouragement to do better

COSMIC retreat October 2005 UCAR COSMIC Project Office Missing from CDAAC 1.0 TIP instrument processing: Level 1b radiance and pointing information file Level 2 inversion (combined with RO) Difficult to debug until we have real data COSMIC level 0 to 1 code Open Loop capability Dual cluster functionality Operator interface to payload SOH data TBB processing capability Issues Reported at 2004 Retreat:

COSMIC retreat October 2005 UCAR COSMIC Project Office Missing from CDAAC 1.0 TIP instrument processing: Level 1b radiance and pointing information file Level 2 inversion (combined with RO) Difficult to debug until we have real data COSMIC level 0 to 1 code Open Loop capability Dual cluster functionality Operator interface to payload SOH data TBB processing capability Issues Reported at 2004 Retreat:

COSMIC retreat October 2005 UCAR COSMIC Project Office TIP State of Health ● TIP State of health extraction done: tipSOH.pl ● Generates tipSOH data type which contains: ● High voltage status plot ● Filter heater status plot ● Command and response summary ● Based on IDL code from Scott Budzien of NRL

COSMIC retreat October 2005 UCAR COSMIC Project Office TIP Radiance code ● TIP pointing location from attitude (part of tipRadiance.pl) is complete. ● Awaiting code from NRL for converting TIP raw counts to radiances.

COSMIC retreat October 2005 UCAR COSMIC Project Office Missing from CDAAC 1.0 TIP instrument processing: Level 1b radiance and pointing information file Level 2 inversion (combined with RO) Difficult to debug until we have real data COSMIC level 0 to 1 code Open Loop capability Dual cluster functionality Operator interface to payload SOH data TBB processing capability Issues Reported at 2004 Retreat:

COSMIC retreat October 2005 UCAR COSMIC Project Office COSMIC zero to one code ● cosmicZero2one.pl complete ● Tested with several sample data files from NSPO, including 9 dumps during the recent early orbit simulation. ● Operates on either USN download files or NSPO raw satellite dump files ● Generates: ● High rate opnGps GPS data ● Low rate BINEX GPS data ● TIP raw data ● Satellite attitude data, including rough position and velocity and inertial quaternions ● GOX state of health extract

COSMIC retreat October 2005 UCAR COSMIC Project Office Missing from CDAAC 1.0 TIP instrument processing: Level 1b radiance and pointing information file Level 2 inversion (combined with RO) Difficult to debug until we have real data COSMIC level 0 to 1 code Open Loop capability Dual cluster functionality Operator interface to payload SOH data TBB processing capability Issues Reported at 2004 Retreat:

COSMIC retreat October 2005 UCAR COSMIC Project Office Open loop zero to one code ● JPL open loop code was highly experimental ● Written in perl and C ● Used a 300+ Megabyte intermediate ASCII file ● Slow—10 to 15 minutes to run ● Updated this code into one clean efficient C code and a perl wrapper ● More compact (80 Megabyte) binary output file ● Much faster—1 minute or less to run ● Consolidated JPL Bjfmtl.c code. Now one version works for COSMIC, CHAMP and SACC open loop ● New high rate data format: opnGps. Replaces atmGps BINEX file which was hard to adapt to new open loop requirements

COSMIC retreat October 2005 UCAR COSMIC Project Office Statistics of comparison of the RO inverted N to ECMWF analysis. global - better penetration; - larger standard deviation PLL OL

COSMIC retreat October 2005 UCAR COSMIC Project Office Statistics of comparison of the RO inverted N to ECMWF analysis. tropics - better penetration; - larger standard deviation - smaller negative N-bias PLL OL

COSMIC retreat October 2005 UCAR COSMIC Project Office OL RO signals capture PBL OL PLL

COSMIC retreat October 2005 UCAR COSMIC Project Office OL RO signals capture PBL A,B,C - sea D - sea-land boundary E,F - land

COSMIC retreat October 2005 UCAR COSMIC Project Office Missing from CDAAC 1.0 TIP instrument processing: Level 1b radiance and pointing information file Level 2 inversion (combined with RO) Difficult to debug until we have real data COSMIC level 0 to 1 code Open Loop capability Dual cluster functionality Operator interface to payload SOH data TBB processing capability Issues Reported at 2004 Retreat:

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC Hardware Status ● All Hardware has been purchased ● Primary processing cluster is in place and being tested ● Primary I/O server is running and receiving NR CAN ● Web based data available to collaborators ● Back up processing and I/O servers are in place ● Failover procedures for I/O and processing strings due mid December

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC Operations Status ● Monitoring of I/O server and processing cluster performance in a rudimentary state. ● Complete monitoring scripts, including dial out to pager/cell phone for on call operator due mid December

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC Computer Room Status ● AC power and backup UPS in current room is adequate only for short (15 minute) outages ● Cooling is barely adequate and not redundant ● As a result of extensive analysis and survey of needs a new facility is being prepared. ● Backup generator will guarantee AC power for up to 2 day outage ● Adequate cooling with redundant systems will be available ● UCAR maintenance is preparing the room and will have it available by end of 2005 ● CDAAC clusters will move to new facility Jan 2006

COSMIC retreat October 2005 UCAR COSMIC Project Office Missing from CDAAC 1.0 TIP instrument processing: Level 1b radiance and pointing information file Level 2 inversion (combined with RO) Difficult to debug until we have real data COSMIC level 0 to 1 code Open Loop capability Dual cluster functionality Operator interface to payload SOH data TBB processing capability Issues Reported at 2004 Retreat:

COSMIC retreat October 2005 UCAR COSMIC Project Office Operator Interface - Leo Dumps - Added COSMICT (COSMIC test) mission and changed display to show results for up to 6 LEO Ids instead of only one. - Leo dump queries now show both: dumps created with links to dump info, and dumps that were not created due to an error.

COSMIC retreat October 2005 UCAR COSMIC Project Office Operator Interface - Leo Dumps - A click on the dump ID will dynamically generate a status page with: * Refractivity statistics for all occultations * Number of files produced of the following types: o atmPhs o atmPrf o avnPrf o wetPrf * Failure percentages and reasons

COSMIC retreat October 2005 UCAR COSMIC Project Office Operator Interface - Fiducial Information - The main screen allows display for the current day, with the option to view up to ten days back. The display shows: * Average Latency Map for a given day for CosHrf and IgsHrf files (New) * Fiducial RealTime Data Inventory (New) * Hour count of CanHrf, CosHrf, and IgsHrf files for the present day (New) * MP1_AVG and MP2_AVG (GPS data quality indicators), EPOCHS_SUM (the total number of epochs for this day) and NOBS_SUM (the total number of observations for this day) for each site.

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC - Internal - Besides The Operator Interface, the following sections have been updated for the next release: - Operator Manual: - Source Code data flow diagrams:

COSMIC retreat October 2005 UCAR COSMIC Project Office Missing from CDAAC 1.0 TIP instrument processing: Level 1b radiance and pointing information file Level 2 inversion (combined with RO) Difficult to debug until we have real data COSMIC level 0 to 1 code Open Loop capability Dual cluster functionality Operator interface to payload SOH data TBB processing capability Issues Reported at 2004 Retreat:

COSMIC retreat October 2005 UCAR COSMIC Project Office TBB processing ● No progress

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC activities ● Improved fiducial network and data handling ● Development of Data Bit Systems ● Single difference processing greatly reduces processing time and data volume ● Receiver Testing ● Data QC ● Ionospheric processing

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC activities ● Improved fiducial network and data handling ● Development of Data Bit Systems ● Single difference processing greatly reduces processing time and data volume ● Receiver Testing ● Data QC ● Ionospheric processing

COSMIC retreat October 2005 UCAR COSMIC Project Office FIDUCIAL SITE STATUS ● Real time processing ● ~30 stations in a real time stream from NR Canada (canHrf) ● Three netRS receivers in South Pacific (cosHrf) ● The balance of stations we use for real time (total = 50) are fetched from the IGS (CDDIS) based on latency and coverage ● Script rateIgsHrf.pl determines which igsHrf sites to fetch. This list will be updated monthly. will add “ntrip” sites

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC activities ● Improved fiducial network and data handling ● Development of Data Bit Systems ● Single difference processing greatly reduces processing time and data volume ● Receiver Testing ● Data QC ● Ionospheric processing

COSMIC retreat October 2005 UCAR COSMIC Project Office Bit Grabber System 10 systems built Currently deployed User guide on web Data already proven beneficial Considering patent for use of data bits in RO processing

COSMIC retreat October 2005 UCAR COSMIC Project Office Host agreed Host needed Global Bit Grabber network

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC activities ● Improved fiducial network and data handling ● Development of Data Bit Systems ● Single difference processing greatly reduces processing time and data volume ● Receiver Testing ● Data QC ● Ionospheric processing

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC POD Status Near real-time POD demonstrated with CHAMP and current ground network data Single-Difference processing with 30-sec GPS clocks to be used for excess phase generation in Post-Processing and Near Real-Time Include CDAAC functionality to handle LEO S/C maneuvers Include COSMIC satellite GPS antenna phase centers and possibly variations Modify Bernese software to process data from 2 POD antennas Install occultation prediction software New Results: Future Work:

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC GPS Ground Networks Near Real-Time Post-Processing

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC Zenith Troposphere Delay Results Post-Processing Near Real-Time Note different color scales

COSMIC retreat October 2005 UCAR COSMIC Project Office Post-Processed and NRT CHAMP Orbit Position Overlap Results (vs. JPL) (Daily 24 hour arcs)(Arcs for every CHAMP dump) Near Real-TimePost-Processing

COSMIC retreat October 2005 UCAR COSMIC Project Office Computation of excess atmospheric delay Double Difference Advantage: Station clock errors removed, satellite clock errors mostly removed (differential light time creates different transmit times), general and special relativistic effects removed Problem: Fid. site MP, atmos. noise, thermal noise Single Difference LEO clock errors removed use solved-for GPS clocks Main advantage: Minimizes double difference errors

COSMIC retreat October 2005 UCAR COSMIC Project Office Spectral Analysis of GPS Clocks Analyzed spectrum of difference of 2 GPS clocks, with 1-sec data 30-sec fiducial sampling adequate for RO data processing 30-sec Period

COSMIC retreat October 2005 UCAR COSMIC Project Office DD vs. SD Post-Processing Status ( ) Single-Difference with 30-sec GPS clocks Double-Difference with 1-sec ground data

COSMIC retreat October 2005 UCAR COSMIC Project Office DD vs. SD Near Real-Time Status ( ) Single-Difference with 30-sec GPS clocks Double-Difference with 1-sec ground data

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC activites ● Improved fiducial network and data handling ● Development of Data Bit Systems ● Single difference processing greatly reduces processing time and data volume ● Receiver Testing ● Data QC ● Ionospheric processing

COSMIC retreat October 2005 UCAR COSMIC Project Office Receiver Testing (a) Planning to test all COSMIC flight receivers outdoors on roof of NSPO Working with BRE engineers and JPL on test plan and procedure Tests planned for 1st week of Nov. ‘05 Will set up web site for on-line analysis of test data in real- time (to be used by test engineers) Tests contain some risk Interpretation / consequences of bad data Tight schedule to fix/change receivers

COSMIC retreat October 2005 UCAR COSMIC Project Office Receiver Testing (b) Need to keep the tests simple !! Check SNRs and compare to SNRs from UCAR roof tests (and to commercial reference receiver) Check cycle slips - to see if receiver remains locked Need to test 4 antennas / satellite

COSMIC retreat October 2005 UCAR COSMIC Project Office NSPO Roof Pre-Survey Satellite Visibility L1 signal to noise ratio (dbhz)

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC activities ● Improved fiducial network and data handling ● Development of Data Bit Systems ● Single difference processing greatly reduces processing time and data volume ● Receiver Testing ● Data QC ● Ionospheric processing

COSMIC retreat October 2005 UCAR COSMIC Project Office CDAAC QC and error estimation progress since last year Roam QC has been fine-tuned and new QC checks have been introduced to remove errors related to: Truncation of L1 when this signal is lost Truncation of L2 when this signal is lost Truncation of radio holographic bending angle profile ‘Bad’ signals (tracking errors and 1 second spike (CHAMP) Dynamic error estimation for bending angles and refractivities below 20 km – errors estimated without using external data Both QC and error estimation techniques have not yet been tested with open loop data – this will be done in the coming future

COSMIC retreat October 2005 UCAR COSMIC Project Office Old processing vs. new processing - deviations from ECMWF refractivity CHAMP August 2002

COSMIC retreat October 2005 UCAR COSMIC Project Office Global Statistics of dynamically estimated measurement errors CHAMP August 2002

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC activities ● Improved fiducial network and data handling ● Development of Data Bit Systems ● Single difference processing greatly reduces processing time and data volume ● Receiver Testing ● Data QC ● Ionospheric processing

COSMIC retreat October 2005 UCAR COSMIC Project Office Ionosphere processing 1) TIP:- running NRL’s radiance code and from the orbits and attitude info calculate the pointing location on the Earths surface. 2) Absolute TEC:- Cycle slip correction, QC, and DCB calibration. 3) Profiles:- Abel transform + hopefully some kind of an attempt to correct for the horizontal gradients via a model or TEC maps. 4) Scintillation maps for the 50 Hz data (generated by Bill S).Unclear what we will do with TBB. Plan to have an antenna on the roof.

COSMIC retreat October 2005 UCAR COSMIC Project Office Other new CDAAC features ● Daemon management infrastructure: daemons.pl ● Technique for syncing up systems with CVS: cvsSync.pl ● Improved file transfer technique 'QCP' helps TACC latency ● New var1d code - tuned so that results are consistent with radio occultation refractivity ● COSMIC/CHAMP 6 LEO test case and operator interface: got rid of many bugs in handling multiple LEOs. Now can display status of six satellites. ● Scripts in place to compare results between two clusters (leo-a and leo-b or TACC and CDAAC) on an ongoing basis. ● Improved Bernese V5.0 menu utility speeds processing up up to 5 fold

COSMIC retreat October 2005 UCAR COSMIC Project Office Summary ● CDAAC operational version is almost done ● Next delivery to NSPO/CWB delayed until Nov ● Remaining Tasks ● Tip code from NRL needs to be incorporated ● TBB - need to install a receiver on roof get data ● Decoding / display of real-time payload health and diagnostics data needs to be improved ● Display of results - show&tell needs to be finalized ● ….