Flagging: When Good Data Go Bad Scott Schnee & Amy Kimball Nov 9, 2011 Interferometry Discussion Group

Initial Flagging Shadowing Pointing Errors – Reported – Unreported Observing Log Other obvious problems

Initial Flagging Shadowing – Issue at low elevations – Issue for compact arrays In CASA for an ALMA data set: flagdata(vis=‘vis.ms’, mode=‘shadow’, diameter=12.0) In MIRIAD for a CARMA data set: csflag(vis=‘vis.ms’, carma=true)

Initial Flagging Pointing Errors – Reported (e.g. wind) – Unreported (e.g. pointing model) In MIRIAD for a CARMA data set: uvflag(vis=‘vis.ms’, select=“pointing(5,100000)”, flagval=flag) uvflag(vis=‘vis.ms’, select=“el(85,90)”, flagval=flag)

Initial Flagging Observing Log Many observatories will note problems that affect the system for part or all of a track. This could be weather-related or hardware- related, and often requires that some data be flagged.

Initial Flagging Other obvious problems Source Sys Temps (K) URANUS C C L1451MM L1451MM C C L1451MM L1451MM C

Initial Flagging Other obvious problems Tsys plots from NGC 3256 ALMA CASA Guide

What to look for? Plots of amplitude and phase vs time and channel Iterate over – Antenna – Spectral window – Source Make plots of bandpass and gain calibrators first – Easy to find bad data of a bright point source – Hard to find bad data of a faint extended source

What to look for? Smoothly varying phases and amplitudes can be calibrated Discontinuities can not be calibrated Features in the calibrators that may not be in the target data can cause problems

Amplitude vs Time From TW Hydra ALMA Guide Color: Polarization One spectral window (spw) plotted

Locating the Bad Data in plotms Draw a box around the suspected bad data.

Locating the Bad Data in plotms Click locate and CASA will send information about the data to the logger.

Locating the Bad Data in plotms Bad data can be flagged by pressing this button or using the flagdata tast at the CASA prompt.

Locating the Bad Data in plotms Flagger’s remorse can be corrected by unflagging good data

From TW Hydra CASA Guide Brown and Green show phase calibrators Orange shows TW Hya Amplitude vs Frequency Birdies

Amplitude vs Frequency Edge Channels

Data that should be flagged

Amplitude vs Frequency Spectral Lines in Bandpass Calibrator From TW Hydra Band 7 Guide Spectral line in Titan

Phase vs Time Phase Jumps First batch of data Second batch of data From Antennae ALMA CASA Guide

Possible Flagging Technique 1.Flag obviously bad data 2.Calibrate the data 3.Flag newly found bad data 4.Re-calibrate 5.Iterate (3, 4) or declare victory

After Calibration, Look Again From NGC 3256 ALMA CASA guide Amplitude vs Time, after calibration

Sage Advice From Rick Perley to a much younger Scott Schnee: “When in doubt, throw it out.”

Online flags (from Steve Myers and Josh Marvil) Tbuff (before August 2011) Quacking sometimes necessary

Known RFI at science.nrao.edu

A flagging/calibration recipe EXAMINE bandpass/flux calibrator(s) FLAG bandpass/flux calibrators APPLY bandpass/flux calibration to itself APPLY bandpass/flux cal to phase cal sources EXAMINE phase cal sources FLAG phase cal sources APPLY phase calibration to itself APPLY bandpass/flux/phase cal to targets EXAMINE targets FLAG targets Iterate Repeat as necessary

Flagversion control in CASA Beware of applycal!

Recognizing low-frequency RFI Average all times: RFI visible on the shortest baselines One EVLA user’s recommendation: flag these frequencies on all baselines at all times

Individual timeranges can be bad