1. NRAO-SKA Future Collaboration Meeting

2. Common Astronomy Software Applications - CASA
CASA is the primary data reduction package for ALMA and the VLA.
Used daily in operations both by individual PIs and Pipeline
Flexible general purpose package can be applied to many telescopes

3. Deployment Supported Operating Systems
Latest 2 OSX releases (currently and 10.12)
Red Hat Enterprise Linux 6 and 7
Most standard Linux distributions can run this version
Deployment in a variety of environments
Desktop systems
Small institutional clusters
NRAO and Academic
Amazon Web Services
National Supercomputing Centers via XSEDE
NRAO-SKA Discussion May 4, Charlottesville VA

4. CASA Use Cases Calibration and imaging of interferometric data
VLA, ALMA, ATCA, GMRT,…
Calibration and imaging of Single Dish data
ALMA, Nobeyama, …
Commissioning and testing activities
But lacking some TelCal functionality
Holography, Baseline Determination
Currently rely on AIPS (VLA) and CLIC (ALMA)
Imaging and Calibration Algorithm Research and Development
Automated Data Processing Pipelines
NRAO-SKA Discussion May 4, Charlottesville VA

5. Development Team
CASA is developed by an international team led by NRAO
Current worldwide effort is 31.5 FTEs:
Annual investment 5.5 M$/ yr
17 FTE from NRAO (~3 M$)
5.5 FTEs are dedicated to automated pipeline production for ALMA and VLA
An additional ~11 FTEs support the CASA project
Project Scientists
Scientific Testing
Verification Testing
Education & Helpdesk
NRAO-SKA Discussion May 4, Charlottesville VA

6. Still in early days, expect workflow to evolve.
Development Process
CASA is moving to Git for version control
Using a feature branching workflow
CI builds on the feature branches (two levels of testing)
Validation performed on the branch prior to pull request
Using the Atlassian Suite under an open source license
Jira for issue tracking
BitBucket as Enterprise Git Service
Locally hosted at NRAO (not cloud hosted)
Bamboo as CI system
Testing of trunk and feature branches on commit
Still in early days, expect workflow to evolve.
NRAO-SKA Discussion May 4, Charlottesville VA

7. Other Collaborations
JIVE Project funded by Black Hole Cam working on VLBI capabilities (~1FTE)
Fringe fitting
Necessary interchange for VLBI operations
Tsys etc.
ATCA ( < 1 FTE)
Continued support for ATCA in CASA.
Development of CASA guides for ATCA data
GMRT (<1 FTE)
Dedicated support person identified and integrated into NRAO helpdesk.
MeerKAT:
CASA Interoperability, Beam Patterns, Holography
IDIA (South Africa):
HPC and Algorithmic for wide field, wide band, full stokes imaging.
10 Nov 2016: ASIAA ALMA Regional Center

8. CASA Codebase 2 M Lines of Code AIPS / IRAF: ~1 MLOC
Astropy: kLOC
NRAO-SKA Discussion May 4, Charlottesville VA

9. COCOMO-II Model Effort Estimates bases on COCOMO-2 default values
kLOC
FTE-y
ASAP 81 25
gcwrap
406
136
code
1051
370
casacore
476
160
Effort Estimates bases on COCOMO-2 default values
ASAP is deprecated (functionality now exists in code)
gcwrap has large amounts code generated (probably over-estimate)
code + casacore is 530 FTE -Years
Estimated value ~ 92 M$
NRAO-SKA Discussion May 4, Charlottesville VA

10. CASA Architecture: CASA Core
Maintained on GitHub as open source project
Primary contributors: NRAO and ASTRON
Contains:
Array, Matrix, etc.: Templated mathematical objects
Quanta, Measures: Quantities and Frames
Table System: I/O Subsystem
Image Definition: Specialization of Table System
Measurement Set Definition: Primary data model
FITS: Conversion to/from FITS Standard
UVFITS: Conversion to/from FITS Standard
FITS IDI
Common Lingua Franca for many projects, do not have a complete list of stakeholders.
ASAP
Code
GCWrap
CASA Core
NRAO-SKA Discussion May 4, Charlottesville VA

11. CASA Architecture: Code
Majority of CASA package
Repository for algorithms and implementation
Contains:
CASA Core
ASAP
GCWrap
Code
Visibility Access
Table Plot
VI/VB/ TVI
Plot MS
HPC / Parallelization
Viewer
Flagging Algorithms
Single Dish
Heuristic
Replacement for ASAP
Deterministic
Image Analysis
Fillers
Calibration Framework
SDM, MIRIAD, ATCA
Imaging Subsystem
NRAO-SKA Discussion May 4, Charlottesville VA

12. CASA Architecture: gcWrap
Interface layer for functionality in lower layers
IPython (code completion, syntax highlighting)
AIPs like interface (list of function parameters)
Much of this is code generated
Definition is in XML
Tools
Swig binding to top level C++ objects.
Stateful and powerful, but not very transparent
Tasks
Atomic tasks based on tool functionality
Wraps most common functionality into easy to use form
Interface for most “standard” users
CASA Core
ASAP
Code
GCWrap
NRAO-SKA Discussion May 4, Charlottesville VA

13. CASA Architecture: ASAP
ATNF Spectral line Analysis Package
Single dish capabilities in a pure python package
ScanTable format (casacore tables)
Maintained by NAOJ (in loose communication with ATNF) to support Total Power analysis for ALMA
Multiple antenna support
Tsys support, improved baseline subtraction algorithms, sideband separation.
Deprecated in favor of C++ implementation now part of code
Performance issues
Interoperability with other CASA tasks in ALMA pipeline
CASA Core
Code
GCWrap
ASAP
NRAO-SKA Discussion May 4, Charlottesville VA

14. CASA Architecture: CARTA
Cube Analysis and Rendering Tool for ALMA
Developed by ALMA Development Project
PI: Erik Rosolowsky
Now primary development at ASIAA CASA Development Center
NRAO and IDIA (ZA) are collaborating
CARTA
GCWrap
ASAP
Code
CASA Core
Full featured replacement for the CASA Viewer
JavaScript based(both desktop and remote server configurations)
Plug-in architecture for expansion
First release this summer
Continued development and expansion by collaboration
NRAO-SKA Discussion May 4, Charlottesville VA

15. CASA Architecture: Pipeline
Calibration pipelines exist for both ALMA and VLA
Mature design, continuing to refine heuristics
Imaging Pipelines not as mature
ALMA Imaging Pipeline in operations use.
QA / Reference Images, science quality in some cases.
VLA Imaging Pipeline under development
VLASS specific heuristics currently
Design
All Python, not “just” a script
Atomic tasks
Parameters exposed to users
“Inputs” come from context
Weblog for User Feedback
Pipeline
CARTA
GCWrap
ASAP
Code
CASA Core
Cycle 2
Cycle 3
Cycle 4
ALMA Calibration Pipeline Performance
NRAO-SKA Discussion May 4, Charlottesville VA

16. Pipeline Weblog
NRAO-SKA Discussion May 4, Charlottesville VA

17. ALMA Imaging Pipeline
Cycle 4 Imaging Pipeline result towards NGC5728 that was delivered as part of a PI science observation. No human intervention was used to generate the image. Full calibration and imaging took 17 hours.
211 pipeline runs (plus 13% are fully manual)
NRAO-SKA Discussion May 4, Charlottesville VA

18. CASA Performance There is a historical reputation that CASA is slow.
Benchmarking against AIPS has shown:
Imaging in CASA is comparable to faster
Individual tasks are comparable to AIPS in execution time.
CASA used to use [still does a bit] disk for caching between tasks
Vi/Vb rework designed to allow on the fly operations now in place
SDM to MS Filler still under-performing; change of maintainer, will be addressed in the coming year.
New CASA Liaison hire will repeat benchmarking identify current hotspots and document.
NRAO-SKA Discussion May 4, Charlottesville VA

19. Calibration Parallelization
CASA has implemented data parallel processing using MPI.
Most of the “long” calibration tasks are now parallel.
Calibration solve steps not parallelized.
Not a dominate term
SDM to MS Filler currently under refactoring, parallization planned in future.
This is tested at moderate levels of parallelization.
Should scale to large data cases, but may want to parallelize the solve side as well.
NRAO-SKA Discussion May 4, Charlottesville VA

20. Imaging Parallelization
Two approaches to parallelization:
Cube: By plane (actually by chunk of planes)
Continuum: Data parallel for gridding
Serial in deconvolver step; but scales by image size not data
NRAO-SKA Discussion May 4, Charlottesville VA

21. Development Drivers ALMA and VLA Operations
Must continue to support evolution of existing telescopes
New observing modes, capabilities, depths, issues
VLA Sky Survey
All sky S-band survey in 3 epochs
OTF-I observing mode
Automated Calibration and Imaging Pipelines
Science Ready Data Products
Interactive and Automated production of Science Quality products.
Will push for more automated heuristics in CASA and Pipeline
Next Generation Very Large Array
Push to larger N array (but still storing visibilities)
New imaging regime with new challenges
NRAO-SKA Discussion May 4, Charlottesville VA

22. Road Map Unified Visibility Access Scratchless Operation
Parallelization
Documentation Update
Full Polarization Imaging
Time Variable Sources
Build System Rework
CASA-Core Refresh
RFI Mitigation
User Interface Redesign
NRAO-SKA Discussion May 4, Charlottesville VA

23. Opportunities
Global Radio Astronomy benefits from having a general purpose package, able to process the data from many observatories.
But this flexibility comes at the expense of optimization and complexity.
CASA does not currently support several modes that SKA will require
Holography
Multi-Beam Systems
Baseline Determination
All Sky Imaging
The data volume challenges faced by SKA would require CASA to become more sophisticated in:
Parallelization
Co-Processor Use
I/O and Persistence
Error Trapping Workflow Mgmt
Common Visualization and Analysis tools improve access and implementation time for new approaches and algorithms.
NRAO-SKA Discussion May 4, Charlottesville VA

24. Challenges
CASA is in an operational mode, used daily for VLA and ALMA (as well as other telescopes).
User Support requires slow migration of interfaces and modes
Changes which are disruptive and do not benefit the users directly are very unpopular.
Must continue to support desktop systems (OSX and Linux) as well as cluster environments.
CASA as implemented may not be a terribly good fit for the SDP design.
Probably use C++ libraries directly, not passing through python layer
Adding general user interface to these capabilities would be extra work
CASA parallelization to date has been relatively narrow breadth, SKA requires a very different model, not clear if these scale.
NRAO-SKA Discussion May 4, Charlottesville VA

25. SKA - CASA Interaction
How we organize the interaction depends on the level of use of CASA in SKA.
Minor additional capabilities for CASA:
CASA is used for commissioning activities (acceptance testing etc)
SDP developed in parallel for operations processing
Moderate involvement from SKA:
CASA used for commissioning and operations until SDP ready, CASA must support early operational modes.
Separate SDP package developed but not expected to be available for early operations.
Major SKA investment in CASA:
CASA developed as primary operations package for SKA
NRAO-SKA Discussion May 4, Charlottesville VA

26. Some Bad Ideas Some possible collaboration models that will not work:
Fork the CASA Project, SKA branch develops independently
Two teams developing independently on the same codebase
Could work for small modifications, but not at any scale
Two teams working on different long lived branches, merging to a common codebase and rectifying differences.
Merge is unlikely to occur, will foster “us vs. them” mentality
NRAO-SKA Discussion May 4, Charlottesville VA

27. Proposed Organizational Structure
SKA Head of Software
NRAO DMS AD
CASA Product Manager
SKA-SDP CASA Deputy
SKA-SDP Group Lead
NRAO Group Lead
ESO Group Lead
ASIAA Group Lead
NAOJ Group Lead
SDP-ZA
SDP-EU
NRAO-SKA Discussion May 4, Charlottesville VA

28. Proposed Organizational Structure
NRAO DMS AD
CASA Product Manager
SKA-SDP Group Lead
NRAO Group Lead
ESO Group Lead
ASIAA Group Lead
NAOJ Group Lead
SDP-ZA
SDP-EU
NRAO-SKA Discussion May 4, Charlottesville VA

29. Proposed Organizational Structure
NRAO DMS AD
CASA Product Manager
SDP-EU
SDP-ZA
NRAO Group Lead
ESO Group Lead
ASIAA Group Lead
NAOJ Group Lead
NRAO-SKA Discussion May 4, Charlottesville VA

30. Cross Team Interaction
CASA uses sub-projects to organize major efforts
Level of formality depends on scope of projects
Structure and design of each sub-project can be customized
Staff from different geographical areas can be matrixed in to work on a project.
Some examples of Sub-Projects:
CASA Core Refresh (or this could be broken into multiple bits)
High Throughput Computing (again, probably better to be focused)
Streaming Processing Interface
NRAO-SKA Discussion May 4, Charlottesville VA

31. www.nrao.edu • science.nrao.edu
The National Radio Astronomy Observatory is a facility of the National Science Foundation operated under cooperative agreement by Associated Universities, Inc.
• science.nrao.edu