Challenges in user provided quality control information for real time temperature and ocean currents Jeff Donovan and Vembu Subramanian University of South Florida College of Marine Science SECOORA Data management meeting, March 2006

Quality Assurance (QA) vs. Quality Control (QC)‏ Quality Assurance – procedures performed prior to instrument deployment to support the return of best possible quality data Quality Assurance – procedures performed prior to instrument deployment to support the return of best possible quality data Setup and testingSetup and testing CalibrationCalibration Proper maintenanceProper maintenance Quality Control – procedures/processes applied to the (real time) data returned from the instrument Quality Control – procedures/processes applied to the (real time) data returned from the instrument Format checksFormat checks Proper timingProper timing Range checksRange checks Other checksOther checks Neighbor checks Neighbor checks Climatology checks (regional and site specific) ‏ Climatology checks (regional and site specific) ‏ Model Comparisons Model Comparisons SECOORA Data management meeting, March 2006

QA Case Study: Two Years of Frustration RD Instruments Workhorse ADCP freshly calibrated and brand new instrumentsfreshly calibrated and brand new instruments local compass calibration passedlocal compass calibration passed all RDI suggested pre-deployment tests passedall RDI suggested pre-deployment tests passed instrument set up and parameters verifiedinstrument set up and parameters verified return of data verified through serial cable and satellite downlinkreturn of data verified through serial cable and satellite downlink format of returned data verifiedformat of returned data verified instrument deployedinstrument deployed All user QA procedures performed and passed Results? SECOORA Data management meeting, March 2006

Results: ADCP begins power cycling shortly after deployment ADCP begins power cycling shortly after deployment Occurred in 70% of our deployments over the past two years (not problematic prior to that) ‏Occurred in 70% of our deployments over the past two years (not problematic prior to that) ‏ Power cycling throws off the system clock, so real time data transmissions are also affectedPower cycling throws off the system clock, so real time data transmissions are also affected Huge recent loss of both real time and instrument stored dataHuge recent loss of both real time and instrument stored data RD Instruments response RD Instruments response Claimed that we were the only group reporting this type of problemClaimed that we were the only group reporting this type of problem Replaced PIO $2500/instrument, did not correct the problemReplaced PIO $2500/instrument, did not correct the problem Suggested we had power related issues on all affected mooringsSuggested we had power related issues on all affected moorings Final problem diagnosis Final problem diagnosis Our mooring technician contacted other groups (GoMoos, CaroCOOPS, etc.) and found that they were also having similar problemsOur mooring technician contacted other groups (GoMoos, CaroCOOPS, etc.) and found that they were also having similar problems RD Instruments finally researched problem and acknowledged that there was a bug in their firmware (v16.21) ‏RD Instruments finally researched problem and acknowledged that there was a bug in their firmware (v16.21) ‏ Conclusion Conclusion Would have saved us lots of time and effort as well as reduced our loss of data if the problem was looked into more carefully when first reportedWould have saved us lots of time and effort as well as reduced our loss of data if the problem was looked into more carefully when first reported Instance where best practices of QA were followed, but loss of data still occurredInstance where best practices of QA were followed, but loss of data still occurred SECOORA Data management meeting, March 2006

QC of real time sea surface temperature data Easy Checks Easy Checks Range checks (manufacturer and local “sanity” checks) ‏Range checks (manufacturer and local “sanity” checks) ‏ Missing dataMissing data Time continuity checkTime continuity check Rate of change of sampleRate of change of sample Data formatData format Difficult Checks Difficult Checks Neighbor checks (instruments not on same platform) ‏Neighbor checks (instruments not on same platform) ‏ Climatology check (regional and site specific) ‏Climatology check (regional and site specific) ‏ Comparison to model outputComparison to model output We use in-situ data to initialize and to provide checks on models. At this time many coastal ocean models are not advanced to the point where they can provide QC checks on the data. We use in-situ data to initialize and to provide checks on models. At this time many coastal ocean models are not advanced to the point where they can provide QC checks on the data. SECOORA Data management meeting, March 2006

Neighbor Check Buoy spacing makes this difficult to apply Buoy spacing makes this difficult to apply Slow moving or stalled fronts can create large differences between nearest neighbors Slow moving or stalled fronts can create large differences between nearest neighbors Bigger problem with measurements which change more rapidly (eg. Wind, air temp.) ‏ Bigger problem with measurements which change more rapidly (eg. Wind, air temp.) ‏ If they do not match, which one is assumed (in)correct? If they do not match, which one is assumed (in)correct? SECOORA Data management meeting, March 2006 Virmani and Weisberg, June 2006

Neighbor Check (cont.)‏ Virmani and Weisberg, June 2006 Water near shore cools quicker in fall/winter and warms quicker on spring/summer Water near shore cools quicker in fall/winter and warms quicker on spring/summer If you are going to attempt this comparison, you must be very careful in choosing the neighbor you use If you are going to attempt this comparison, you must be very careful in choosing the neighbor you use EC3 and NA2 mooring separated by only 8.8 km, and in February the SST can differ by 2-3 deg. C EC3 and NA2 mooring separated by only 8.8 km, and in February the SST can differ by 2-3 deg. C SECOORA Data management meeting, March 2006

Regional Climatologies All climatologies are not created equal, be very careful which one you choose This panel shows anomalies of SST for Jan. thru Mar. Look at NCEP.vs. ECMWF or ECMWF.vs. Oberhuber Differences as large as 6 deg. C between them near shore Using climatologies tends to flag all “events” as questionable SECOORA Data management meeting, March 2006 J. Virmani, Phd. Thesis, 2006

Site Specific Climatologies SECOORA Data management meeting, March 2006 Need long time series to compute a high quality, site specific climatology Need long time series to compute a high quality, site specific climatology Example: C17, C18, and C19 have been in the water for two years. Example: C17, C18, and C19 have been in the water for two years. High number of hurricanes in 2004 and 2005 have passed near or directly over these mooringsHigh number of hurricanes in 2004 and 2005 have passed near or directly over these moorings Both SST and currents will show a large bias based on these eventsBoth SST and currents will show a large bias based on these events

The production of local climatologies are possible and desirable but they require long time series. Below are examples of “climatological (3-yr) currents” superimposed on climatological (5-yr) SST. SECOORA Data management meeting, March 2006

Required Required Pitch/Roll/HeadingPitch/Roll/Heading UVW – Vertical/Horizontal VelocityUVW – Vertical/Horizontal Velocity Echo AmplitudeEcho Amplitude CorrelationCorrelation Recommended Recommended BIT StatusBIT Status Battery Voltage (v1 & v2) ‏Battery Voltage (v1 & v2) ‏ Water TemperatureWater Temperature PressurePressure TimestampTimestamp Std. Deviation of speedsStd. Deviation of speeds Transmit currentTransmit current Correlation coefficientCorrelation coefficient Speed of soundSpeed of sound Signal to noise ratioSignal to noise ratio Surface (bottom) reflectionSurface (bottom) reflection Percent good solutions/Percent good pingsPercent good solutions/Percent good pings Error VelocityError Velocity QARTOD III suggestions for QC of real time ocean current data SECOORA Data management meeting, March 2006

Implications of QARTOD III in-situ ocean current recommendations: A COMPS Example COMPS sites transmit real time data in ASCII format via GOES with two different transmitter speeds COMPS sites transmit real time data in ASCII format via GOES with two different transmitter speeds 1200 baud, 15 second window, 1.5 second guard band top and bottom1200 baud, 15 second window, 1.5 second guard band top and bottom 100 baud, 60 second window, 5 second guard band top and bottom100 baud, 60 second window, 5 second guard band top and bottom COMPS sites send back a combination of meteorology and in-water sensor data back in the same transmission with our in-situ ocean current data COMPS sites send back a combination of meteorology and in-water sensor data back in the same transmission with our in-situ ocean current data Volume of data is a problem Volume of data is a problem Maximum of 1770 ASCII characters can be sent with 1200 baud transmitterMaximum of 1770 ASCII characters can be sent with 1200 baud transmitter Maximum of 567 ASCII characters can be sent with 100 baud transmitter. This is used on very shallow sites, and two transmissions are made from each siteMaximum of 567 ASCII characters can be sent with 100 baud transmitter. This is used on very shallow sites, and two transmissions are made from each site SECOORA Data management meeting, March 2006

A COMPS Example (cont): COMPS Site C16 COMPS Site C16 Return ADCP currents, in-water temperature and salinity, and position information all in a single GOES transmissionReturn ADCP currents, in-water temperature and salinity, and position information all in a single GOES transmission ADCP set up with PD8 command to return ASCII dataADCP set up with PD8 command to return ASCII data PD8 command returns time stamp, pitch/roll/heading, temperature, sound speed, BIT status, bin number, direction, speed, east/north/vertical/error velocities, and echo amplitude PD8 command returns time stamp, pitch/roll/heading, temperature, sound speed, BIT status, bin number, direction, speed, east/north/vertical/error velocities, and echo amplitude Our data logger parses each ensemble and extracts time stamp, speed, direction, and temperatureOur data logger parses each ensemble and extracts time stamp, speed, direction, and temperature Speed and Direction (75, 1m bins) require 1200 characters Speed and Direction (75, 1m bins) require 1200 characters Time stamp and temperature require 27 characters Time stamp and temperature require 27 characters In water T/S, position information and other control characters require 425 characters In water T/S, position information and other control characters require 425 characters Leaves 118 spare characters Leaves 118 spare characters Some of the other required/recommended data is available in the PD8 formatSome of the other required/recommended data is available in the PD8 format Pitch/roll/heading and BIT status would require 24 characters Pitch/roll/heading and BIT status would require 24 characters East and North velocity (75, 1m bins) could replace speed and direction East and North velocity (75, 1m bins) could replace speed and direction Vertical and error velocity (75, 1m bins) would require 1200 characters Vertical and error velocity (75, 1m bins) would require 1200 characters Four beams of echo amplitude (75, 1m bins) would require 1200 characters Four beams of echo amplitude (75, 1m bins) would require 1200 characters This is a total of 2424 additional ASCII characters This is a total of 2424 additional ASCII characters We would need an additional two GOES transmissions per site to just send back all of the required and part of the recommended parameters under our current design We would need an additional two GOES transmissions per site to just send back all of the required and part of the recommended parameters under our current design This could not be done with the older 100 baud transmitters, thus making them useless This could not be done with the older 100 baud transmitters, thus making them useless SECOORA Data management meeting, March 2006

Based on these required checks, all of the COMPS ocean current data would have to be marked as either questionable or no check performed Based on these required checks, all of the COMPS ocean current data would have to be marked as either questionable or no check performed Will data flagged this way still be made available through search engines and/or the SEACOOS web site?Will data flagged this way still be made available through search engines and/or the SEACOOS web site? How will this look to a funding agency if a project is not providing ANY data with a QC flag of good?How will this look to a funding agency if a project is not providing ANY data with a QC flag of good? How were these requirements designed? How were these requirements designed? Since we are talking about real time data, some thought needs to given to the different methods of transmitting the data and the bandwidth associated with these methodsSince we are talking about real time data, some thought needs to given to the different methods of transmitting the data and the bandwidth associated with these methods Can we meet all the requirements? Can we meet all the requirements? Yes, but it will take a complete redesign of our entire arrayYes, but it will take a complete redesign of our entire array What is the time frame to put this in place?What is the time frame to put this in place? This will take a substantial investment in both time and moneyThis will take a substantial investment in both time and money Where does this funding come from?Where does this funding come from? What happens to our data availability while we are redesigning our array?What happens to our data availability while we are redesigning our array? A COMPS Example (cont): SECOORA Data management meeting, March 2006

Conclusions QA/QC procedures are a very important part of the process of disseminating real time data QA/QC procedures are a very important part of the process of disseminating real time data All data providers should participate in this process to the best of their ability and strive to provide the best quality data possible All data providers should participate in this process to the best of their ability and strive to provide the best quality data possible It will be very difficult to decide on one set of “required” procedures that will be able to be completely implemented by the entire community It will be very difficult to decide on one set of “required” procedures that will be able to be completely implemented by the entire community We need to stop short of making it too difficult or cost prohibitive for data providers to comply with the proposed requirements of data QC We need to stop short of making it too difficult or cost prohibitive for data providers to comply with the proposed requirements of data QC It is essential that we facilitate making coastal ocean measurements where they are needed scientifically. Access to sufficient bandwidth to accommodate the transmission of both data and the ancillary information necessary to support the proposed QC requirements should not play a role in this decision. It is essential that we facilitate making coastal ocean measurements where they are needed scientifically. Access to sufficient bandwidth to accommodate the transmission of both data and the ancillary information necessary to support the proposed QC requirements should not play a role in this decision. SECOORA Data management meeting, March 2006