Future CABB Data Reduction Mark Wieringa | Science Data Processing | Australia Telescope National Facility ATUC - 11 th June 2013 CSIRO ASTRONOMY AND SPACE.

Slides:



Advertisements
Similar presentations
S o f t w a r e D e f i n e d R a d i o
Advertisements

Science Operations Phil Edwards 30 October 2012 Space for collaborator logos [delete instructions before use] CSIRO ASTRONOMY & SPACE SCIENCE.
Imaging MS MIAPE Working Document Helmholtz Institute, Munich, April 16 th 2012.
Imaging MS MIAPE Working Document Helmholtz Institute, Munich, April 16 th 2012.
Corruption of Visibilities Editing Calibrators Mark Wieringa - CSIRO.
CSIRO; Swinburne Calibration & Editing Emil Lenc University of Sydney / CAASTRO CASS Radio Astronomy School 2014 Based on lectures given.
1 st post launch SCIAMACHY calibration & Verification Meeting L1b Astrium Friedrichshafen – Germany 24 July 2002 First Level 1b Spectral Calibration analysis.
The evaluation and optimisation of multiresolution FFT Parameters For use in automatic music transcription algorithms.
Ray Norris, Jamie Stevens, et al. Deep CABB observations of ECDFS ( Extended Chandra Deep Field South )
ATNF Scientific Software – an overview Mark Wieringa – ATNF/Operations/SCA.
Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Whoever North American.
Image Analysis Jim Lovell ATNF Synthesis Imaging Workshop May 2003.
Image Analysis Jim Lovell ATNF Synthesis Imaging Workshop September 2001.
BLAH BLAH BLAH nvvbnvbnvbncvncn. Abstract A complete sample of point source emissions in the spiral type galaxies NGC300 and M31 have been catalogued.
AEOS Telescope + Image-Slicer/ Spectrograph Data Reduction and Results for June 8, 2006.
Prototype SKA Technologies at Molonglo: 3. Beamformer and Correlator J.D. Bunton Telecommunications and Industrial Physics, CSIRO. Australia. Correlator.
BLAH BLAH BLAH nvvbnvbnvbncvncn. Abstract A complete sample of point source emissions in the spiral type galaxies NGC300 and M31 have been catalogued.
Darren Baskill Timing mode spectral correction MOS Timing mode spectra problem.
Astro Space Locator Imaging Software Alexey Rudnitskiy, Victor Zuga On behalf of Astro Space Center (ASC LPI) Department Of Data Processing Alexey Rudnitskiy,
CSIRO; Swinburne Error Recognition Emil Lenc University of Sydney / CAASTRO CASS Radio Astronomy School 2012 Based on lectures given previously.
Technologies for Radio Astronomy Graeme Carrad | Assistant Director - Engineering Mark Bowen | Group Leader – Front End Technologies JULY 2012 CSIRO ASTRONOMY.
Frazer Owen NRAO UNM Astrophysics Seminar November 3, Galaxy Evolution and Mechanical Energy Abell 2256 with the EVLA: Data Reduction and Results.
Mark Bowen Acting Theme Leader – Technologies for Radio Astronomy CSIRO Astronomy and Space Science 14 October 2010 Australia Telescope Users Committee.
Phased Array Feeds John O’Sullivan SKANZ 2012 CSIRO Astronomy and Space Science,
Wideband Imaging and Measurements ASTRONOMY AND SPACE SCIENCE Jamie Stevens | ATCA Senior Systems Scientist / ATCA Lead Scientist 2 October 2014.
HIFI Tutorial 3: Getting some basic science out A.P.Marston ESAC 27 June 2013.
Spectral Line VLBI Chris Phillips JIVE The Netherlands Chris Phillips JIVE The Netherlands.
Data Handling and Reduction Jamie Stevens May
CABB Data Reduction Review Summary of recommendations Dave McConnell June 2011.
Australia Telescope 20 GHz survey (AT20G): An Update Ron Ekers (ATNF, PI),Sarah Burke(Haverford-Swinburne), Mark Calabretta (ATNF), Gianfranco De Zotti.
CABB update Philip Edwards Head of Science Operations 14 October 2010.
Science Operations Phil Edwards 11 July 2012 Space for collaborator logos [delete instructions before use] CSIRO ASTRONOMY & SPACE SCIENCE.
ALMA Integrated Computing Team Coordination & Planning Meeting #4 Santiago, November 2014 Telescope Calibration Planning Dominique Broguière.
Wide-field imaging Max Voronkov (filling up for Tim Cornwell) Software Scientist – ASKAP 1 st October 2010.
ATNF Operations Douglas Bock ATUC – 25 NOVEMBER 2014.
Sanjay BhatnagarEVLA Advisory Committee Meeting May 8-9, EVLA Algorithm Research & Development Progress & Plans Sanjay Bhatnagar CASA/EVLA.
Adaptive Filters for RFI Mitigation in Radioastronomy
EOVSA Data and Database System J. McTiernan EOVSA CoDR 25-April-2011.
Herschel Open Time Cycle 1 DP workshop ESAC, March page 1 Spurs in HIFI data Colin Borys.
VLBI2010 Phase Calibration Signal Generator and Broadband Considerations Christopher Beaudoin Research Engineer MIT Haystack Observatory.
Atacama Large Millimeter/submillimeter Array Expanded Very Large Array Robert C. Byrd Green Bank Telescope Very Long Baseline Array Observing Scripts Basic.
FILTERS. Filter The purpose of a filter is to pass signals of certain frequencies,
Recent progress in EVLA-specific algorithms EVLA Advisory Committee Meeting, March 19-20, 2009 S. Bhatnagar and U. Rau.
The ATNF Front End and Back End Groups Presented by Graeme Carrad Assistant Director – Engineering - May 2010 Australia Telescope Users Committee Technologies.
1 Synthesis Imaging Workshop Sept 24-28, 2001, ATCA Narrabri Calibration & Flagging Mark Wieringa  Theory of calibration  Practice - RFI & malfunctions.
ESO, 17 April 2007ESAC meeting1 ALMA offline User Test 5 Silvia Leurini, ESO.
SKA: Configurations and Simulations Ramesh Bhat Colin Lonsdale Roger Cappallo Shep Doeleman Divya Oberoi Joanne Attridge MIT Haystack Observatory.
CABB support and ATCA flux calibration Jamie Stevens ATCA Senior System Scientist 26 May 2010.
1 Tracking tests – Analysis results October 2007 Campaign Panagiota Xydi Acknowledgements: to all FIDEL team Cern, December 09, 2007 AT-MTM.
Мulti-frequency Algorithm at Russian Software Package ASL L. Kogan, S. Likhachev, N. Kardashev, E. Fomalont, F. Owen, E. Greisen.
NHSC HIFI DP workshop Caltech, 7-9 February page 1 Spurs in HIFI data.
10 January 2006AAS EVLA Town Hall Meeting1 The EVLA: A North American Partnership The EVLA Project on the Web
Introduction to Multichannel Image Reconstruction, Image Analysis and Display (MIRIAD)
 History of early Universe; the Epoch of Reionization  Goal: Map the evolution of structure of the early Universe using the Murchison Widefield Array.
First result with PAF on a big single-dish radio telescope X. Deng, A. Chippendale, S. Johnston, G. Hobbs, D. George, R. Karuppusamy ASTRONOMY AND SPACE.
Data Analysis “I have got some data, so what now?”
GUSTO Data Products and Archiving Plan
S. Likhachev (1), L, Kogan(2), E. Fomalont(2), F. Owen(2)
S. Bhatnagar: ASTRON, Dwingeloo, June 29th 2010
Data Reduction and Analysis Techniques
HIFI Hands-on Exercises
Data Reduction and Analysis Techniques
Wide-field imaging Max Voronkov (filling up for Tim Cornwell)
Oxford Algorithms Workshop
1-channel 2-channel 4-channel 8-channel 16-channel Original
Some Illustrative Use Cases
From Russia with love!!!.
Early indications of performance
EVLA Algorithm Research & Development
Australia Telescope Users Committee Technologies Report – October 2011
Presentation transcript:

Future CABB Data Reduction Mark Wieringa | Science Data Processing | Australia Telescope National Facility ATUC - 11 th June 2013 CSIRO ASTRONOMY AND SPACE SCIENCE

Miriad improvements for CABB Cope with increased data size (>8GB) up to channels, image sizes up to Calibration in up to 16 frequency bins (gpcal, selfcal) Solutions interpolated in time and frequency Fixes phase slope across band Removes effect of secondary calibrator spectrum Solves leakage terms vs frequency Better data flagging capabilities New flagging task pgflag with automatic flagging mode (SumThreshold alg.) Blflag improvements – plot vs channel, cope with more data Source fitting with spectral index and curvature (uvsfit, uvfit binning) Frequency dependent primary beam correction Linmos bw parameter, restor mfs option ATUC October 2012

Miriad limitations Imaging algorithms Mfclean – intensity and spectral index term only (limits fractional bandwidth to <50%, errors on edge of primary beam) Clean algorithms dont cope well with extended emission, MEM algorithms dont do MFS Mosaicing – no MFS (mosmem, mossdi), primary beam at fixed frequency No major cycle clean that subtracts from ungridded data – DR limitations No automatic source detection and cleaning Image cubes use single beam plane Ad hoc analysis No tools to get easy access to data from scripts ATUC October 2012

The committee has surveyed the shortcomings of Miriad when faced with the calibration and imaging of recent CABB data. Taking into account the limited resources available, a number of minor improvements to Miriad have been suggested and prioritised, totaling a few months of development. In addition the committee suggests that both the CASA and ASKAPSoft packages are used to supplement the imaging and analysis tasks available in Miriad. Documentation of the process to get data into these packages (and back) and recipes or scripts for further processing should be provided. Report Conclusion ATUC October 2012

CABB data reduction with CASA Latest release: CASA (31 May 2013) Main features of interest: Multi-scale, multi-frequency clean (MSMFS) – now working well for single fields Can use multi-scale clean (but not yet MFS) for mosaics Cube imaging with beam per channel, can clean planes in parallel A-projection – cope with time and baseline dependent primary beam effects (no MFS yet) Drawbacks: No native filler for rpfits yet. Use FITS export/import or new uv2ms task in linux Miriad Imaging can be slow – factor compared to Miriad EVLA has all of CABBs problems and more (non-coplanar, beam squint) Solutions will appear in CASA over time (research topic) CASA/CABB data reduction workshop Jamie Stevens will organize this if there is sufficient community interest Can now calibrate & image CABB Stokes I, continuum & spectral line data in CASA Polarization still needs testing ATUC October 2012

Data Reduction Recommendations Load data into Miriad for calibration and flagging Produce first pass images If quality of images limited by extended emission spectral structure primary beam structure Apply calibration, Convert to MeasurementSet Produce images with CASA MSMFS algorithm A-projection algorithm (need Voltage patterns for ATCA antennas) ATUC October 2012

Plans for Miriad Bug fixes Minor features, calibration improvements No major work on imaging Provide easy transfer to CASA ATUC October 2012

Plans for CASA Workshop ATCA CASA Data Reduction Cookbook Start contributing ATCA/CABB specific work Primary beam models Calibrator models Contribute ASKAPSoft imaging algorithms Re-implement in CASA ATUC October 2012

Thank you CSIRO Astronomy and Space Science Mark Wieringa Science Data Processing t wwww.atnf.csiro.au CSIRO ASTRONOMY AND SPACE SCIENCE