CSIRO; Swinburne Calibration & Editing Emil Lenc University of Sydney / CAASTRO www.caastro.org CASS Radio Astronomy School 2014 Based on lectures given.

Slides:



Advertisements
Similar presentations
Future CABB Data Reduction Mark Wieringa | Science Data Processing | Australia Telescope National Facility ATUC - 11 th June 2013 CSIRO ASTRONOMY AND SPACE.
Advertisements

3mm Testing in July 2001 Tony Wong ATNF Epping CA Forum, 6 Sep.
Corruption of Visibilities Editing Calibrators Mark Wieringa - CSIRO.
CABB Observations Preparations and Observing Mark Wieringa.
Interferometric Spectral Line Imaging Martin Zwaan (Chapters of synthesis imaging book)
A rough guide to radio astronomy and its use in lensing studies Simple stuff other lecturers may assume you know (and probably do)
Image Analysis Jim Lovell ATNF Synthesis Imaging Workshop May 2003.
Twelfth Synthesis Imaging Workshop 2010 June 8-15 Observation Preparation Tool (OPT) Loránt Sjouwerman (NRAO/Socorro)
Interference Daniel Mitchell, ATNF and Sydney University.
1 Synthesis Imaging Workshop Error recognition R. D. Ekers Narrabri, 20 Sep 2006.
1 Synthesis Imaging Workshop Error recognition R. D. Ekers Narrabri, 14 May 2003.
LOFAR Self-Calibration Using a Blackboard Software Architecture ADASS 2007Marcel LooseASTRON, Dwingeloo.
Prospects for High-Frequency Calibration with the SMA Dual-IF/Receiver System Todd R. Hunter, Jun-Hui Zhao (CfA) Sheng-Yuan Liu, Yu-Nung Su, Vivien Chen.
Calibration with CASA Masaaki Hiramatsu 平松正顕 (ASIAA/NTHU) ALMA User Workshop, Feb 8-10, 2010.
CSIRO; Swinburne Error Recognition Emil Lenc University of Sydney / CAASTRO CASS Radio Astronomy School 2012 Based on lectures given previously.
J.M. Wrobel - 25 June 2002 PROPOSALS 1 PROPOSAL WRITING TUTORIAL Outline 30 minutes: Lecture on Generic Issues 60 minutes: Small Groups Write Proposals.
Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Emmanuel Momjian (NRAO)
Random Media in Radio Astronomy Atmospherepath length ~ 6 Km Ionospherepath length ~100 Km Interstellar Plasma path length ~ pc (3 x Km)
Slide 1 25-Jan-10Huib Intema Recent low-frequency developments at NRAO Charlottesville and some other USA institutes.
Survey Quality Jim Condon NRAO, Charlottesville. Survey Qualities Leiden 2011 Feb 25 Point-source detection limit S lim Resolution Ω s Brightness sensitivity.
Tenth Summer Synthesis Imaging Workshop University of New Mexico, June 13-20, 2006 Antennas in Radio Astronomy Peter Napier.
P.Napier, Synthesis Summer School, 18 June Antennas in Radio Astronomy Peter Napier Interferometer block diagram Antenna fundamentals Types of antennas.
Spectral Line VLBI Chris Phillips JIVE The Netherlands Chris Phillips JIVE The Netherlands.
Twelfth Synthesis Imaging Workshop 2010 June 8-15 Advanced Calibration Techniques Mark Claussen NRAO/Socorro (based on earlier self-calibration lectures)
Intraday variability of Sgr A* at radio wavelengths: A Day in the Life of Sgr A* Doug Roberts Northwestern University Adler Planetarium & Astronomy Museum.
ATCA synthesis workshop - May ATCA – Calibration at mm wavelengths Rick Forster University of California, Berkeley Hat Creek Radio Observatory Berkeley-Illinois-Maryland.
Polarization at IRAM Status and Plans S.Guilloteau Laboratoire d’Astrophysique de Bordeaux.
Observing strategies Max Voronkov Software Scientist – ASKAP 28 th September 2010.
CABB Data Reduction Review Summary of recommendations Dave McConnell June 2011.
1 EVLA System Commissioning Results EVLA Advisory Committee Meeting, March 19-20, 2009 Rick Perley EVLA Project Scientist.
Interferometry Basics Andrea Isella Caltech Caltech CASA Radio Analysis Workshop Pasadena, January 19, 2011.
Ninth Synthesis Imaging Summer School Socorro, June 15-22, 2004 Spectral Line II John Hibbard.
Academia Sinica National Taiwan University AMiBA System Performance Kai-yang Lin 1,2 and AMiBA Team 1,2,3 1 Institute of Astronomy and Astrophysics, Academia.
G. Moellenbrock, Synthesis Summer School, 18 June Calibration and Editing George Moellenbrock Why calibration and editing? Formalism: Visibilities,
Sanjay BhatnagarEVLA Advisory Committee Meeting May 8-9, EVLA Algorithm Research & Development Progress & Plans Sanjay Bhatnagar CASA/EVLA.
Observing Strategies at cm wavelengths Making good decisions Jessica Chapman Synthesis Workshop May 2003.
NASSP Masters 5003F - Computational Astronomy Lecture 14 Reprise: dirty beam, dirty image. Sensitivity Wide-band imaging Weighting –Uniform vs Natural.
Tenth Summer Synthesis Imaging Workshop University of New Mexico, June 13-20, 2006 High Dynamic Range Imaging Craig Walker.
Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Radio Frequency Interference.
ASKAP: Setting the scene Max Voronkov ASKAP Computing 23 rd August 2010.
The Australia Telescope National Facility Ray Norris CSIRO ATNF.
Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Observing Scripts Basic.
Recent progress in EVLA-specific algorithms EVLA Advisory Committee Meeting, March 19-20, 2009 S. Bhatnagar and U. Rau.
Calibration You never know – it might be important.
1 Synthesis Imaging Workshop Sept 24-28, 2001, ATCA Narrabri Calibration & Flagging Mark Wieringa  Theory of calibration  Practice - RFI & malfunctions.
Jan URSI1 Fast Switching Phase Compensation for ALMA Mark Holdaway NRAO/Tucson Other Fast Switching Contributors: Frazer Owen Michael Rupen Chris.
Requirements for Single-Dish Holography Parameter Specification Goal Measurement error
Flagging: When Good Data Go Bad Scott Schnee & Amy Kimball Nov 9, 2011 Interferometry Discussion Group.
L. Young, Synthesis Imaging Summer School, 19 June Spectral Line I Lisa Young This lecture: things you need to think about before you observe After.
1 ATNF Synthesis Workshop 2003 Basics of Interferometry David McConnell.
CABB support and ATCA flux calibration Jamie Stevens ATCA Senior System Scientist 26 May 2010.
ATNF Synthesis Workshop - September Single-dish spectral-line observing Lister Staveley-Smith, ATNF  Why use a single-dish?  Analogy with interferometer.
Imaging issues Full beam, full bandwidth, full Stokes noise limited imaging Algorithmic Requirements: –PB corrections: Rotation, Freq. & Poln. dependence,
MWA imaging and calibration – early science results
Direction dependent effects: Jan. 15, 2010, CPG F2F Chicago: S. Bhatnagar 1 Direction-dependent effects S. Bhatnagar NRAO, Socorro RMS ~15  Jy/beam RMS.
A potted outline of the EVN data
Fringe-Fitting: Correcting for delays and rates
following the lectures of Ron Ekers (and others)
Observing Strategies for the Compact Array
Observing and Data Reduction
Polarization Calibration
Max Voronkov Software Scientist – ASKAP 28th September 2010
Observational Astronomy
Observational Astronomy
ALMA Calibration Introduction
Early indications of performance
Basic theory Some choices Example Closing remarks
Non-imaging Analysis and Self-calibration
EVLA Algorithm Research & Development
Molecular Imager: Focal Plane Array
Presentation transcript:

CSIRO; Swinburne Calibration & Editing Emil Lenc University of Sydney / CAASTRO CASS Radio Astronomy School 2014 Based on lectures given previously by Mark Wieringa and John Reynolds See also “Calibration and Editing”, Fomalont E.B., Perley, R.A., Synthesis Imaging in Astronomy II, 1999, vol. 180, p.79

An inconvenient truth ›Atmosphere -Ionosphere -Troposphere ›Antenna/Feed -On-axis gain/sensitivity vs El -Primary beam correction -Pointing -Position (location) ›LNA+conversion chain -Clock -Gain, phase, delay -Frequency response ›Digitiser/Correlator -Auto-leveling -Sampling efficiency -Birdies FT(Observed Visibilities) ≠ Pretty Image ›Measureables -Amplitude -Phase -Delay -Rate -Polarization -Spectrum

The Measurement Equation i j V ij V ij = V pp V pq V qp V qq Ji=Ji= JpJqJpJq

The Measurement Equation Baseline based gain errors Correlation corrections Antenna based: J vis = B G D P B = bandpass G = complex gain D = pol leakage P = receptor pos angle (2x2 matrices) Baseline based additive A ij = noise + RFI + offsets Polarization Conversion Antenna beam + pointing Faraday Rotation Sky Intensity Distribution

Simplified Model V ij obs = (J i  J j ) V ij mod + noise (vectors of 4 polarizations) ›Just look at antenna based gain, leakage and bandpass -Ignore or separate out other terms -Troposphere ›Antenna based terms: J i = B i (v) G i (t) D i (vectors of 2 pol.) ›Solve iteratively, one/two component(s) at a time -Minimize χ 2 =  ij,t  V ij obs (t) - (J i (t) x J j (t)) V ij mod   ›Use calibration sources (calibrators) to simplify the process -Strong point source -Can ignore noise and source structure -Known, stable flux-density -No time dependence, fixes flux scale -Little or no polarization -Can determine instrumental polarization terms easily ›Similar scheme works for more complex sources -Need to build up a model of the field iteratively -See High DR Imaging lecture

Visibility corruption ›RFI – interference -Transmitters, Lightning, Solar, Internal RFI ›Antenna/Receiver/Correlator failures -no signal, excess noise, artificial spectral features ›Bad weather -effect gets worse for higher frequency (very low frequency too) -decorrelation, noise increase, signal decrease (opacity) ›Shadowing -one antenna (partially) blocked by another

Flagging / Editing ›Rule 1. Don’t be afraid to throw out data -Corrupted data can reduce the image quality significantly -Effect of missing data (even 25%) is often minor and easily corrected in deconvolution ›Rule 2. Flag data you know is bad early -Save your sleuthing skills for the hard stuff -See “Error recognition” talk ›Rule 3. Use shortcuts where possible -Detailed visual inspection of all data is rapidly becoming impossible -Collapse, average, difference & automate using scripts

Flagging / Editing ›1st pass: use on-line flags (automatic) -Flags when antennas are off source or correlator blocks offline. ›2nd pass: Use the observing logbook! Saves lots of time later. -Note which data is supposed to be good & discard data with setup calibration, failed antennas, observer typos etc. ›3rd pass: Use automatic flags -Correlator birdies, Common RFI sources (options=birdie, rfiflag) -Shadowed data: select=shadow(25) -Data with bad phase stability: select=seeing(300) ›4th pass: Check calibrators - plot amp-time, phase-time, amp-freq -investigate outliers & flag, flag source as well if you can't trust data ›5th pass: (After calibration) Inspect & flag source data -Use Stokes V to flag data with strong sources

Flagging / Editing RFI: 1-3 GHz

Flagging / Editing PGFLAG - Interactively flag, tune params - Automatic flagging from script SumThreshold flagging - Subtracts smooth background - Clips on running mean in x and y

ATCA Calibration Sequence ›Observatory – done after reconfiguration -Bulk delay (cable lengths) -Baseline (antenna location) – good to 1-5 mm -Antenna Pointing – good to 10”-20” ›User -Schedule preparation (observing strategy) -dcal/pcal/acal: “Real-time” first-pass approximation -Post-observation calibration

Calibration at reconfiguration ›antenna pointing (global pointing model derived from sources in all Az/El directions) -generally correct to better than 10", occasional 20" error single antenna -may need reference pointing with nearby cal above 10 GHz ›baseline lengths (relative antenna positions) -generally correct to better than 1-2 mm (depending on weather) -error significant at 3mm - correct phase with nearby calibrator ›global antenna delay (bulk transmission delay in cables)

Calibration – Schedule Planning ›Observe primary calibrator (cm), Uranus(mm) min, to calibrate the absolute flux scale -cm/1934: can also solve for polarization leakage and bandpass -mm: -Observe separate bandpass calibrator -Use secondary for polarization leakage ›Observe secondary calibrator (close to target) -1-2 min every min -Atmospheric, instrumental phase variation, -System gain variations; optional: solve leakage, bandpass ›Observe pointing calibrator (above 10 GHz) -a POINTing scan every minutes

ATCA / VLA Calibrator List ›Ideal secondary calibrator is strong, small (θ<λ/B max ) and close to the target (<15°) -ATCA + VLA lists ~1000 sources -Calibrator database lets you make the optimal choice ›Primary flux calibrators are also stable with time: PKS , PKS ›Above 20GHz, the planets are essentially the only primary flux calibrators -all bright compact sources seem to vary at high frequency -Planets not ideal – resolved on longer baselines / seasonal variation but recent work has been done to constrain these.

Calibration – Starting up ›Calibration done at start of observation: -Delay calibration -Correct residual path length for your particular frequency & correlator setup -Fixes phase slope across band -Amplitude & Phase -Equalize gains, zero phases, sets Tsys scale -helps to detect problems during observation. -Polarization -zero delay & phase difference between X & Y feeds -uses noise source on reference antenna to measure phase. -generally correct to a few degrees at 3-20 cm

Initial Array Calibration ddcal pca l acal

Calibration – During Observation ›Observations of secondary calibrator [+ pointing cal] -Tsys correction -estimates system temp from injected noise -corrects for e.g., ground pickup & elevation, but not for atmospheric absorption -At 3mm: use Paddle scan calibration instead ›Calibration data recorded during the observation: -Tsys – system temperature -XY-phase difference on each antenna -(experimental) Water Vapour Radiometer path length

Calibration – Post Observation ›Primary flux calibrator : “bootstrapping” to secondary calibrator ›Solve for polarization leakages (cross-terms) ›Use secondary calibrator to correct (“straighten”) the phases ›Use primary, secondary or other strong point source as bandpass calibrator

Calibration – Bandpass

Calibration – Secondary gains

Calibration – Wideband ›Gain, Phase and Calibrator flux vary across a 2GHz band ›Two approaches possible in Miriad: -Divide and conquer [uvsplit maxwidth=0.256] -Split data into 4-16 narrow bands, calibrate independently -Solve in frequency bins [gpcal nfbin=8] -Solve independently, interpolate solutions -Advantages: -Interpolation fixes phase slope -Quicker / less bookkeeping ›Imaging – combined (PB issue) or separate (SN issue) -CASApy has better imaging options

Primary Calibration 6.5GHz4.5GHz Calibrated plot of all channels – Imaginary vs Real Unpolarized; Variation of calibrator flux with Frequency

Secondary Calibration GHz Uncalibrate d BP, pol Calibrated in 8 freq bins Freq Averaged Calibrated

Calibration Recipe ›Select primary calibrator -Solve for complex gain vs time -Solve bandpass gain vs frequency -Solve polarization leakage (crosstalk between feeds) ›Select secondary calibrator(s) -Apply bandpass and leakage from primary -Solve for complex gain vs time -Bootstrap absolute flux scale (from primary) ›Select sources of interest -Apply bandpass and leakage from primary and complex gains from secondary -Use calibrated data in subsequent imaging and analysis

Real “Primary” Flux Calibration

Absolute Flux Calibration 1Jy = W/m 2 /Hz how??

Real “Primary” Flux Calibration

The Big Four Transferred to ‘secondary’ cals 3C138, , Hydra A Cas. A Cyg. A Vir. A Tau. A

Summary of Reduction Steps ›For ATCA/CABB data we still mostly use MIRIAD (CASA for power users) ›Load the data from the archive format (RPFITS) ›Apply ‘logbook flags’ and check for bad data on calibrators ›Calibrate primary calibrator (G,B,D), transfer to secondary (B,D) ›Calibrate secondary calibrator (G), transfer to source (G,B,D) ›Flag bad data on source (PGFLAG, BLFLAG) ›Analyze data (imaging, statistics, source fitting)

Acknowledgements ›This talk is based on: -Mark Wieringa version of talk (2001, 2008, 2012) -John Reynolds’ version of this talk (2003,2006) -ASP Conference Series Vol. 180, p.79 – available online