ALMA Pipeline Heuristics - Interferometry - Frederic Boone, LERMA Lindsey Davis, NRAO Heiko Hafok, MPIfR John Lightfoot, UKATC Dirk Muders, MPIfR Christine Wilson, McMaster Friedrich Wyrowski, MPIfR Luis Zapata, MPIfR

The Mission After commissioning and early science, deliver pipeline to reduce data automatically Publishable quality? Trustworthy Easy to use / develop SFI, mosaic, single dish

Development Capture expertise Using Casapy tools, can use tasks Incremental

Overview Python program, recipe driven oSeries of stages oDisplay data ‘views’, detect and flag bad data, search for best calibration method oFlagging improves MS quality, calibration method produces best results possible for that dataset oLastly, produce data products; final calibrations, cleaned cubes Output in HTML

Stages - 1 Calculate and display a result view=BandpassCalibration display=ComplexSliceDisplay Calculate bandpass calibration Display as amplitude/phase v channel view=FluxCalibration(BandpassCalibration) display=ComplexSliceDisplay Calculate bandpass calibration Use it in calculation of flux calibrated gains Display as amplitude/phase v time

view=CleanImage(BandpassCalibration, FluxCalibration, sourceType=‘GAIN’, ) display=SkyDisplay Calculate bandpass calibration Use it in calculation of flux calibration Apply calibrations Produce a cleaned image Display as an image on the sky

Results data ‘views’ BandpassCalibration UnNormalisedBandpassCalibration GainCalibration FluxCalibration GroupSplineFluxCalibration CleanImage MosaicCleanImage

Stages - 2 Calculate ‘view’, flag, display view=GainCalibrationSNR(BandpassCalibration) flagger=ImageFlagger(rules=[{‘rule’:’min abs’, ‘limit’:2.0}] display=ImageDisplay Calculate bandpass calibration Use it in calculation of gains, export gain SNR  Apply flagging rules, flag the MS Display as a greyscale image with colour coded flags

MS & CalTable data ‘views’ AmplitudeDeviationPerBaseline PhaseDeviationPerBaseline BandpassCalibration FluxCalibration GainCalibrationSNR ClosureError

view=ClosureError (CleanImage, BandpassCalibration, FluxCalibration, outputDataAxes=‘ANTENNA’, iterateAxes=‘CORR’, dataCompressOperations=[‘abs’, ‘median’], sourceType=‘GAIN’) flagger=SequenceFlagger(rules=[…]) display=SliceDisplay Calculate clean image CORRECTED_DATA - MODEL_DATA [corr, antenna1, t, antenna2] -> [antenna] Assemble list from data belonging to each antenna then compress by taking abs, median  Look for abnormal antennas, flag the MS Display with flags

Displays SliceDisplay / ComplexSliceDisplay matplotlib ImageDisplaymatplotlib SkyDisplaymatplotlib CalPlotDisplayplotCal

Flaggers TAQLFlagger autocorrelation PdB Gibbs channels SequenceFlagger outlier in chunk high outlier low outlier max abs min abs too many flags ImageFlagger high outlier low outlier max abs min abs too many flags BandpassEdgeFlagger VLA edge template PdB edge template

The Future User Test 6 ? Get some real ALMA data! Polarization, transfer of calibrations between SpW Parallelization