1. 1 JPSS Engineering Review Board (ERB) Sea Surface Temperature (SST) Product Configuration Change Request (CCR) Evaluation 9 May 2014

2. 2 Agenda ● Review of the CCR ● JPSS PSE Assessment –Methodology –Review of Assessment ● Recommendations ● Next Steps

3. 3 SST CCR Evaluation Overview ● In August 2013, the JPSS Program Scientist submitted a Configuration Change Request (CCR) to JPSS Program Systems Engineering (PSE) recommending that the program Sea Surface Temperature (SST) product only be produced, validated and sustained within NDE. –The Program Scientist notes that the NOAA critical operational user community, namely the National Ocean Service (NOS) and National Marine Fisheries Service (NMFS) are currently using the NDE Advanced Clear-Sky Processor for Oceans (ACSPO) SST. NOAA users prefer the legacy ACSPO algorithm. No operational users for the IDPS SST product have been identified and research community users (identified through CLASS) have not indicated a preference for either product when questioned. –Potential cost savings may be obtained through suspension of calibration, validation and sustainment activities on the IDPS SST product. Reference the NJO CM Form_SST_final.docx

4. 4 PSE Assessment: Methodology ● Generated an abbreviated trade analysis, working with experts and developers of both products, the user community and the JPSS Ground Project ● Evaluated15 distinct criteria, comparing both implementations only for the relevant category, culminating in a criteria-specific evaluation and a final recommendation –Cost Impact –Schedule Impact –Latency Impact –Attribute Performance –Legacy Continuity –Product Assurance/Risk –Validation Maturity –User Preference ● PSE understands this is not necessarily a comprehensive list and requests feedback, comments and discussion regarding this methodology –Product Interdependency –Transportability/Interoperability –Maintenance Impact –Evolution Potential –Direct Readout Support –Instrument Development Support –Data Availability

5. 5 PSE Assessment: Summary Reference SST-CCR-Evaluation_update may 2014.docx Evaluation CriteriaRecommendation Cost ImpactACSPO SST calibration, validation, sustainment and development costs less Schedule ImpactNone for either algorithm Latency ImpactIDPS SST has lower latency Attribute PerformanceBoth are expected to meet spec; ACSPO currently performing better Legacy ContinuityACSPO SST is legacy Product Assurance/RiskACSPO SST is assured to the level of confidence of current NOAA operations; IDPS SST is assured to NASA 7120.5 requirements Validation MaturityBoth products are at an acceptable validated status; ACSPO SST is operational within NDE User PreferenceNOAA users prefer ACSPO SST Product InterdependencyNone, though must confirm with IDPS Ocean Color team for certain Transportability/ InteroperabilityACSPO SST is more interoperable Maintenance ImpactsNo additional resources are required; funds would need to be reallocated to NDE for cal/val and maintenance Evolution PotentialACSPO SST offers evolution potential Direct Readout SupportNeither is supported for direct readout Instrument Development SupportNeither impacts instrument development Data AvailabilityBoth are available through NDE; only the IDPS SST is currently archived

6. 6 PSE Assessment: Review ● Cost Impact The cost of singular SST algorithm calibration, validation and sustainment is approximately $11,000,000 in either implementation to support STAR work on SST. NJO will incur an additional cost for calibrating, validating and sustaining both implementations of the SST algorithms. However, as one team would support both algorithms, the cost of supporting both amounts to an additional 2 people on the cal/val team staff, totaling approximately $2,750,000 over the lifetime of the mission, a significant cost avoidance to the NJO program. Additional costs will be incurred as the IDPS code is changed, however. SST Cost Comparison spreadsheet. ● Schedule Impact No schedule impact is expected by executing the PSE recommendations for this CCR. If the CCR is executed, some impact to the schedule for delivery of the IDPS may be incurred due to code changes (as discussed in the product interdependency section). ● Latency Impact The IDPS implementation of SST is available at lower latency to the operational users based on PSE accounting of latency allocation; these numbers represent a maximum possible latency and true product delivery timing is expected to be less for both products. However, users have accepted both the aforementioned latencies as part of the JPSS Level 1 Requirements Document and understand the possible delay they face in obtaining only the ACSPO SST product. Non-operational users who will access the product through CLASS will see no impact. Based on previous user acceptance of the ACSPO SST latency, NJO will assume no risk in implementing this CCR. Reference SST-CCR-Evaluation_update may 2014.docx

7. 7 PSE Assessment: Review ● Attribute Performance Both algorithms are expected to meet threshold level 1 requirements, despite lack of funding for validation on ACSPO SST within NDE. There is slight risk to the JPSS program in executing PSE recommendation as there is currently no official plan to validate ACSPO SST explicitly, but this can be mitigated by funding the validation activities. NOTE: ACSPO SST attribute performance has not been officially evaluated by OSD at this time but Alexander Ignatov has provided statistics on algorithm performance for both implementations. ● Legacy Continuity Execution of the PSE recommendations for the CCR will provide NOAA legacy algorithm continuity through ACSPO SST; there is no risk to accepting the CCR. ● Product Assurance/Risk NJO will incur a slight risk associated with product assurance if the CCR recommendations from PSE are executed. The ACSPO SST algorithm has not been developed with NPR 7150.2A rigor, but instead has been developed using current NOAA development procedure. PSE evaluates that this risk is small as the NOAA development procedure in actuality follows CMMI Level 3 and Class B/C rigor. This risk is small as NDE ACSPO SST product will be able to meet science performance standards developed under a CMMI level 3 compliant systems. Users of this legacy algorithm have accepted any risks associated with the CMMI level 3 development previously as ACSPO SST for AVHRR is the current NOAA operational product. Reference SST-CCR-Evaluation_update may 2014.docx

8. 8 PSE Assessment: Review ● Validation Maturity Execution of the CCR recommendations from PSE poses no risk to the program with validation maturity, assuming full funding for ACSPO SST cal/val is provided to STAR. ACSPO SST is currently operational within NDE. ● User Preference If NJO executes the CCR recommendations from PSE, the operational user community will maintain continuity of its legacy products, as requested. The non-operational communities within and external to NOAA will be unaffected as no users were identified for the IDPS-based SST algorithm; in the future, this community will be supported only by ACSPO SST data. If ACSPO SST becomes the sole implementation, there will be no discontinuity since amongst the operational and non-operational user community. There are no risks associated with the CCR execution to NOAA users, NASA or DoD or external personnel. ● Product Interdependency The IDPS-implemented SST algorithm maintains the Ocean Color EDR as part of its backscattering and absorption coefficient calculations. The Ocean Color EDR then flows to the Surface Albedo EDR, subsequently. However, changes to the Ocean Color algorithm have been made and can be swiftly implemented within IDPS to remove the SST dependency, thus alleviating any downstream interdependcy associated with SST. Therefore, execution of the CCR will not have any downstream repercussions if the recommendations above are implemented. Additionally, both algorithms can be used to help address and characterize any issues with the upstream VIIRS SDR. As the STAR team who leads cal/val and sustainment activities on both algorithms is the same, the communication between the SST and VIIRS SDR teams is already in place and will not depend on either implementation. Reference SST-CCR-Evaluation_update may 2014.docx

9. 9 PSE Assessment: Review ● Transportability/Interoperability ACSPO SST has demonstrated that it is both transportable and interoperable, posing no risk to the program if the CCR recommendations from PSE are executed. ● Maintenance Impact Both systems currently maintain their algorithms in the processes highlighted above, so execution of the CCR recommendations from PSE would reduce required DPA and Raytheon resources associated with the IDPS implemented SST product. ACSPO SST maintenance would not be impacted but does require funding as part of cal/val. ● Evolution Potential Currently only ACSPO SST offers and has demonstrated evolution potential, so there is no impact to evaluation potential. ● Direct Readout Support Neither algorithm is currently implemented within the DB product baseline and any support for SST algorithms via DB is done informally through usage of the VIIRS SDR. The JPSS Program Scientist queried the NOAA SST user community of their interest in a formal SST algorithm available from DB. They agreed on the importance of this and noted that the NOAA legacy algorithm, ACSPO SST, is their preferred DB implementation. A formal letter is being drafted and will be sent to the JPSS program echoing this response. Reference SST-CCR-Evaluation_update may 2014.docx

10. 10 PSE Assessment: Review ● Instrument Development Support There is no impact to instrument development support in executing the PSE recommendations in this CCR evaluation. ● Data Availability Both products are available to non-NOAA users operationally through ESPC and through CLASS for longer latency. The IDPS-implemented SST will continue to be archived until the ACSPO SST data are archived, ensuring that no gap in JPSS SST data will exist in the archive. If the CCR recommendations from PSE are executed, there program will incur no risk. Reference SST-CCR-Evaluation_update may 2014.docx

11. 11 Recommended Implementation ● Based on the 15 criteria evaluation, JPSS PSE recommends proceeding with the implementation of this change: –Provide sole product generation responsibility to NDE –Suspend all calibration, validation and development work on the SST algorithm within IDPS –Discontinue all IDPS SST product subscriptions –Discontinue future IDPS SST archival only when ACSPO SST products are archived (Fall 2013) ● In executing the CCR, NJO will save approximately $2.7 M through the program life cycle through suspension of SST calibration, validation and development activities within IDPS ● There will be no impact to the NOAA (and broader) user community since ACSPO SST products will be available operationally and through the NOAA archive. Reference SST-CCR-Evaluation_update may 2014.docx

12. 12 Next Steps ● Assessment recommends the follow-on action items: 1.Confirm once more with the IDPS Ocean Color team that the IDPS SST product is not needed. 2.The JPSS ground team to evaluate what should be done with the SST code in IDPS and obtain a cost evaluation of potential alteratives. 3.Kathryn to provide costing information to the ERB members upon request. ● Recommendation of ERB to NJO Program Control Board: Concurrence with the SST CCR to reallocate sole production responsibility to the ACSPO SST algorithm within NDE

13. 13 Backup

14. 14 Documentation ● CCR submitted by Program Scientist –NJO COM Form_SST_final.docx –Reallocation of level 1 SST EDR to NDE 072913.docx –SST_ACSPO_IDPS_Comparison_v01.pptx ● JPSS PSE evaluation –SST-CCR-Evaulation_update may 2014.docx –SST cost comparison_final.xlsx (furnished upon request to Federal personnel)