OVSA Expansion Software Overview Gordon Hurford Kickoff Meeting NJIT 25-Oct-2010

Monitor and Control Operator Data Packaging Processor DSPU Antennas, receivers, analog subsystems Interim Data Base Data selection, averaging, reformatting & calibration Burst identification Archive Databases ~6 GB/day Users Quick look & metadata products Map generation Tohban Map display Light curve, spectra generation NJIT GH/JM+ NJIT OVSA Expansion Software and Data Handling 23-Oct-10 OVSA-specific CASPER-based Miriad-based RHESSI-based Light curve, spectra display IDL Shell Browser Hi-Level Analysis

Monitor and Control Operator Data Packaging Processor DSPU Antennas, receivers, analog subsystems Interim Data Base Data selection, averaging, reformatting & calibration Burst identification Archive Databases ~6 GB/day Users Quick look & metadata products Map generation Tohban Map display Light curve, spectra generation NJIT GH/JM+ NJIT OVSA Expansion Software and Data Handling 23-Oct-10 OVSA-specific CASPER-based Miriad-based RHESSI-based Light curve, spectra display IDL Shell Browser Hi-Level Analysis

DPP Time-Multiplex Architecture Correlator Frequency averaging RFI excision Frequency averaging RFI excision Time-independent calibration & Formatting Parallel outputs on separate networks Correlator cycles addresses Interim Database + Scaleable + Decouples correlator & DPP design issues + Can trade hardware for code optimization Digital Packaging Processor ~450 MB/s ~1 MB/s 40 GB/day

OVSA Software Task Organization DGAdvisory Committee Data Base Management JM Data Analysis Software GH DPP definition GH Hardware-embedded SW NJIT Array control & real time display NJIT Routine Calibration & Analysis GH Non-solar Analysis CIT Special calibration Analysis NJIT DPP implementation NJIT Housekeeping data NJIT QL / metadata JM Pipelined database creation JM IDL/Miriad shell JM User interface & displays JM Miriad Scripting JM Stephen White Testing NJIT Assisted by NJIT 11-Feb-2010

Implementation Philosophy (1) Implementation is VERY manpower-limited  Prioritization is vital  Maximize use of existing packages  Adaptation of RHESSI IDL-based user interface, display & database systems  Miriad analysis package

Implementation Philosophy (2) Highest priority goals: –To have documented software enabling external users to conveniently do some science with OVSA observations by Sept 30, 2013 –To have basic software tools in place to support hardware development

Implementation Philosophy (3) Highest priority goal: To have documented software enabling external users to conveniently do some science with OVSA observations by Sept 30, 2013 Examples of lower priority goals: –Fine-scale RFI excision –Processing speed –Ability to analyze special cases or compromised events –Implementation of calibration refinements (e.g. polarization) –Integration of high-level analysis tools –Limited support for non-solar observations

Interim Software Milestones (1) Phase 1: 15 months - Dec 31, 2011 Objective: Primitive end-to-end capability to support hardware development & enable demonstration science –Place-holder (pass-through) DPP –Offline processing into Miriad-compatible format –Testing with legacy data –Primitive data base to support test data –Miriad IDL shell – basic development –Some support for calibration analysis –Miriad scripts for calibration and mapping –Limited-feature interface using RHESSI GUI

Interim Software Milestones (2) Phase 2: 15 months - Mar 31, 2013 Objective: Fairly complete but manually-oriented analysis package –DPP supports real-time data packaging and pre- calibration without RFI excision –Improved data base provisions –Improved calibration analysis, application and mapping software –Improved user interfacing. –Preliminary user documentation

Interim Software Milestones (3) Phase 3: 6 months - Sept 30, 2013 Objective: Fully-featured with automated data processing & quick-look generation. –DPP support for RFI excision and flexible frequency averaging –Flare identification and application database generation –Scripts to generate quick look / catalog data. –Scripts to automate data management –Additional bells and whistles. –Basic support for non-solar observations –Improved user documentation..

Next Steps Consensus on overall approach to OVSA software Consensus on calibration strategy Coordination between: –Hardware and software development schedule / needs –NSF- and NASA-funded tasks –OVSA and FASR software Early Definition of selected interfaces –Correlator  DPP –Offline calibration input  DPP –Housekeeping  DPP –Interim database format

extras

DPP Architecture Options Frequency Averaging Baseline Processing Correlator Frequency Averaging By polarization Correlator Frequency Averaging Frequency Averaging Baseline Processing By baseline Correlator Frequency Averaging Frequency Averaging Baseline Processing switched - Not scaleable - Well suited to correlator processing? - Prevents polarization calibration by subchannel + Scaleable -Couples correlator software and/or hardware to DPP speed - Discourages use of ‘canary’ antennas for RFI identification + Fully scaleable + Decouples correlator and DPP performance/design Need speed tests and correlator architecture input