ALMA Integrated Computing Team Coordination & Planning Meeting #1 Santiago, April 2013 Telescope Calibration Planning Dominique Broguiere

ICT-CPM April 2013 Presentation layout n Current status n Work in progress (Recession period) n Plans for next releases (R10.2 and R10.x) n Conclusions

ICT-CPM April 2013 Overall status : TelCal Group n ICT TelCal : 1.3 FTE  Dominique Broguiere, lead, 0.8 FTE  Jean-Christophe Roche, developer, 0.5 FTE n Subsystem Scientist  Baltasar Vila-Vilaro n Deputy Subsystem Scientist  Vincent Pietu (IRAM) n Other contributions  Robert Lucas holography package Scientific support  Alexis Tejeda TelCal contact in JAO group : help to identify Telcal problems Installation of Telcal standalone package on reduction machines

ICT-CPM April 2013 Overall status : Online calibration CalibrationDeliveredPlanning meeting 2012Status 2013 Phase Calx Ampli Calx- Need flux in ASDM - Use model for Solar System objects - delayed until flux issue solved - done Bandpassx Delayx- Use any reference antenna- done Pointingx- improved TotalPower pointing on planets (using disk size) Focusx Atmospherex- Improve efficiency by using sky only scans - implementation done, not tested WVRx- On-line validation required- on-line validation in progress Sideband ratiox- Using side band rejection, needs a little work in side band separation mode - delayed until new sideband separation mode available Skydipx- Needs autocorrelation on horizon source (work-around) - nothing done; not used

ICT-CPM April 2013 Overall status : Offline calibration CalibrationDeliveredPlanning meeting 2012Status 2013 Tower Holographyx Astro Holographyx- Use on-line WVR correction- done Pointing Modelx Antenna Positionsx Primary Beamx- To be done in CASA- done

ICT-CPM April 2013 Overall status (continuation) n Other delivered features :  Include CLIC and RadioPoint in TelCal standalone package  64-bit TelCal standalone libraries for reduction machines  ASDM update changes  New Bulk Data receiver  Performance improvements in some engines (Pointing, Bandpass) and in some offline tasks (asdmplot, RadioPoint)  TelCalSpy improvements n Bug fixes/Tracking issues  a lot of time (~ 50%) spent to analyze problems of fix issues

ICT-CPM April 2013 Work in progress (Recession period) The features for R9.1.x have already been delivered. This page describes the work in progress to finish the validation of some important features as BulkData NT or to fix some issues. n New bulk data system (high priority)  Analyze the possible causes of TelCal slowing down the data flow and fix them  Deliver 64-bit version to remove the 2GB limitation on block sizes n Missing WVR data for arrays without 12m antennas (ACA)  TelCal should be able to use a value of PWV provided by the TelCalParameterTuning interface or retrieve the value determinated by another array, using the TelCalPublisher interface.  For offline reduction, it is necessary to add a parameter in tc_atmosphere task n Bug fixes  CSV-2290 (baseband inversion in ACA atmospheric calibration \)  Etc…

ICT-CPM April 2013 Plan for release R10.2 For this release, currently due for 05 July, a limited number of new features is planned because the priority is to consolidate existing things, to fix bugs and to keep some time to implement improvements proposed by CSV/DSO. n Improvements for off-line mode  Goal : make data reprocessing easier  Enable appending results to the current ASDM for all calibration intents  Use casa rules for the definition of list of parameters (antenna list, bbpairs list, etc…)  Keep the current dataset in memory to avoid time consuming, each time a task is run on the same dataset n Improvements for on-line mode  Priorities to be fixed with CSV (cf TelCal-CSV meeting)  Possible improvements: On-line wvr correction : use the real frequency range of WVR spectral channels

ICT-CPM April 2013 Plan for releases R10.x (x>2) The schedule and the priorities of the R10.x (x>2) features are not strictly defined. They will probably be updated following priority changes or new problems raised. n Improving code performances (computation time)  Becomes more critical as the number of antenna increases  Critical engines Pointing and WVR calibration : CONTROL is waiting for these results  Several possibilities studied by Alexis (June 2011) Code optimization parallelism more efficient compilers (intel compiler)  Current status Pointing and WVR engine performance still acceptable Little work done on this topic due to other priorities –optimization done in some engines –Parallelism (openmp) difficult to introduce due to incompatibilities with some libraries (slatec)

ICT-CPM April 2013 TelCal Performance analysis n Measurements issued from Telcal cpp container log (January-April 2013) n Other time consuming operations in TelCal processing, outside Telcal code :  metaData retrieval from DataCapturer : s  ASDM build from its IDL representation : s EngineComputation timeNumber of antennasComments Pointing 0.5s40 WVR 2s32 Atmosphere 10s32 Full resolution data SideBandRatio 0.4s32 AmpliCal 0.5s32 PhaseCal 6s32 Bandpass 10s32 Delay 6s32 Focus 0.6s 40

ICT-CPM April 2013 Plan for releases R10.x (continuation) n Features dependant on Control schedule  shadowing flag  sideband separation  subarrays (no specific development, just testing)  multi-resolution mode (no specific development, just testing) n Improvements in calibration algorithms  Need analysis and work from astronomers  Topics : on-line WVR correction long baselines high frequencies n Error calculations  Error calculations are missing in some results (.I.e Sideband ratio)  Reduced chi square calculation gives incorrect results in Pointing engine n Flag noisy basebands/ polarization in calibrations  Add the ability to flag some noisy baseband(s) or polarization in calibrations n Documentation : Engines and User guide

ICT-CPM April 2013 ALMA Phasing Project: TelCal involvement n Context  Phasing algorithm will be implemented in TelCal by APP group  Phasing calculation and atmospheric coefficients determination already available in TelCal code n Support of following activities  definition of new ASDM tables  sequence diagram  Phasing algorithm (using existing code of TelCal)

ICT-CPM April 2013 Conclusions n All the required functionalities are in principle already in TELCAL but some features still need to be tested or improved. n Communication between TelCal and other groups is important  CSV/DSO : priorities for new developments or bugs fixing  CONTROL : synchronization of TelCal developments or tests with CONTROL deliveries (scan sequence, shadowing flag, etc…)  ACS : new releases, bulkdata NT  CASA : casa used in offline mode (task interface) and in online mode (solar objects ephemeris)  Etc… n Testing improvements  TelCal Unit tests must be carefully updated and maintained  Integrated tests or simulations are often not useful for TelCal because the data (0 or random values) cannot be used for real calibrations. Would it be possible to use data from existing datasets?