ngVLA Correlator Drivers & Requirements Sean Dougherty (for Michael Rupen) NRC Canada June 26, 2017

Correlator cost drivers SKA CSP MID.CBF Team Meeting, 28th February 2017

The Basics: Correlator Size & Cost A correlator… channelizes: “F-part” cross-correlates: “X-part” To first order: F-part size goes as Ndish * BW * Nbeams X-part size goes as Ndish2 * BW * Nbeams Correlator size is a derived quantity Ndish -- How much collecting area can you afford, and how big are your individual dishes? BW -- Sensitivity trade-offs for narrow/wideband feeds/receivers Nbeams -- Sensitivity vs. FoV trade-offs for single vs. multi-pixel feeds As will be shown, even big correlators are (relatively) cheap F-part= channelizer – scales with # of dishes * bandwidth * number of beams (if multi-beam feeds) X-part= cross-corr’n – scales as # of baselines= # of dishes squared, * bandwidth * # of beams (if multi-beam feeds) FoV= Field-of-view ngVLA Workshop, Socorro, NM June 2017

Scientific Requirements: Correlator Size Number of dishes Driven by sensitivity (ND2), FoV (1/D), uv-coverage (N2) Probably set by total budget (dominated by dish + electronics cost)  200-400 Maximum total bandwidth Continuum sensitivity Line searches & surveys Observing efficiency (get all available information as fast as possible) Ideally limited by receivers & samplers, not by the correlator …but correlator cost does enter in, at the widest bandwidths  20 - 50 GHz Total BW is set by bandwidth ratio of low-noise feeds (1.85:1) …cf. SKA1_Mid split of Band 5 (4.6-13.8 GHz= 9.2 GHz) into Band 5a (4.6-8.51 GHz) & 5b (8.3-15.35 GHz= 7.05 GHz) …ALMA Band 2+3= 67-116 GHz (1.73:1)= 49 GHz ngVLA Workshop, Socorro, NM June 2017

Scientific Requirements: Correlator Size (2) Number of bems Sensitivity vs. field-of-view: so far, hard to make multi-beam feeds competitive SKA1_Mid deemed more effective wide-field instrument than SKA1_Survey Flexible correlators e.g. SKA1Mid.CBF can trade BW for Nbeams  Nbeams= 1 ngVLA Workshop, Socorro, NM June 2017

Latest Big Correlator: SKA1_Mid SKA1_Mid Correlator: 197 dishes, 5 GHz/pol’n, 1 beam Up to ~390k channels (~13 kHz over 5 GHz) Lots of zoom options Robust to RFI: 9+9b correlation, coarse channelization (200 MHz), sample clock offset scheme to avoid correlation of aliased signals FPGA-based (14nm Intel Stratix-10), DDR4 memory, 26G fiber Bottom-up costing based on detailed PBS Vendor quotes for all key items FPGA pricing based on 2021 buy Scaled to 2016 € ~€17M NRE + €3M HW (F-part) + €8M HW (X-part) inc. cont. ~65 kW (F-part) + 175 kW (X-part) Best comparison is to SKA1_Mid Includes ~15% TMT-style contingency Note X-part dominates cost/power for this and larger correlators (N^2) ngVLA Workshop, Socorro, NM June 2017

Scaling to ngVLA: 2021 costing, 2016 € If we were building the ngVLA correlator today: # Antennas Bandwidth/pol’n Nbeams Cost Power 200 5 GHz 1 ~ €28M ~ 240 kW 150 20 GHz ~ €45M ~ 590 kW ~ €60M ~ 960 kW 50 GHz ~€130M ~2400 kW 300 ~€105M ~2000 kW 400 ~€165M ~3300 kW ~€400M ~8300 kW Mid.CBF This is what it would cost if you were to go to production *today*. But that’s not the case… But we are not building the ngVLA today….. ngVLA Workshop, Socorro, NM June 2017

Technology marches on Difficult to forecast! Current indicators say emphasis is on power reduction for 10nm, which maybe appears in 2020 Maybe also for 7nm?, which maybe appears in 2025?? After that ??? 10nm: HW cost x2/3, Power x ½ ? 7nm: HW cost x1/2, Power x 1/3 ??? 10nm: ~ €29M+ €17M (200 dishes), 480 kW ? €101M+ €17M (400 dishes), 1660 kW ?  7nm: ~ €22M + €17M (200 dishes), 320 kW ? €76M + €17M (400 dishes), 1110 kW ? Very uncertain! Guesstimates from Mike Pleasance & Brent Carlson – definitely WAGs (Wild Ass Guesses) but gives some idea of how the FPGA industry is going ngVLA Workshop, Socorro, NM June 2017

Scaling to ngVLA: 2016 €, 2020?/2025?? tech # Antennas Bandwidth/pol’n Nbeams Cost Power 200 5 GHz 1 ~ €28M ~ 240 kW 20 GHz ~ €60M ~ 960 kW 10nm ~ €45M ? ~ 480 kW ? 7nm ~ €39M ?? ~ 320 kW ?? 50 GHz ~€130M ~2400 kW ~ €87M ? ~1200 kW ? ~ €65M ?? ~ 800 kW ?? 400 ~€165M ~3300 kW ~ €120M ? ~1660 kW ? ~ € 93M ?? ~1110 kW ?? ~€400M ~8300 kW ~ €270M ? ~4200 kW ? ~ €200M ?? ~2800 kW ?? Mid.CBF Even 400 dishes & 50 GHz BW is “only” 200M out of 1.5 billion, using 7nm tech -- <15% Power usage is more of a concern Again this assumes no really new technology, and no clever ideas – just do what we’ve designed, using cheaper parts. ngVLA Workshop, Socorro, NM June 2017

Correlator Costs: Conclusions Big uncertainties Correlators are getting cheaper 200-dish, 20 GHz ngVLA correlator would be at most the same fraction of the budget as for SKA1-Mid (~30/325 ~10%) 400-dish, 50 GHz ngVLA correlator maybe 15% of the total Multi-beam, wide BW feeds could change this! Seems silly to cut corners on the heart of the telescope Considerations to pay attention to: ASICs – cost more up front but save power. BUT always a couple technology nodes behind… GPUs – not yet, but keep an eye on them N.B. New technologies and approaches will only make correlators cheaper ngVLA Workshop, Socorro, NM June 2017

Second-order requirements So far we’ve talked about the sheer size & cost of the ngVLA correlator. There are many other requirements on e.g. the spectral resolution and dump time. Here we consider a couple key requirements in that realm. Second-order requirements ngVLA Workshop, Socorro, NM June 2017

Wide-field Continuum Imaging – a demanding use case Beam smearing (chromatic aberration): dfreq/freq= (D/Bmax) * (1/K) Perley 2004: K=4: distortion-free imaging of primary beam (10% loss @ 1st null) Time-averaging losses: dt < Kt * (D/Bmax) Perley 2004: Kt= 3440 (10% drop at 1st null) vs. SKA1_Mid: Kt= 1200 (2% drop at 1st null) D= 18m, Bmax= 300km dfreq/freq= 1.5e-5 * (4/K) = 4.5 km/s  dt~ 0.2 sec Wide-field mapping is one of the most demanding experiments. Following EVLA Memo 64 (Perley 2004), we require the following. Note that the chromatic aberration reqt corresponds to constant velocity resolution over the band  …so if you want, say, 1 km/s for a spectral line search over the full band, just multiply the numbers below by 4.5 . ngVLA Workshop, Socorro, NM June 2017

Wide-field Continuum Imaging (2) K=4, Kt= 3440, D= 18m, Bmax= 300km Ndish= 200 Npol = 4 (full pol’n products) 8 B/vis (as JVLA) Nvis= (Ndish * (Ndish+1)/2) * Npol * Nchan = 80,400 * Nchan Band Nchan Nvis Data rate 1.2- 3.6 GHz 73,242 5.9e9 235 GB/s 3.6-10.8 GHz 1.2- 7.2 GHz 119,452 9.6e9 384 GB/s 70-116 GHz 33,674 2.7e9 108 GB/s Data rates are scary! As is pretty much always true when correlators first come on-line. ngVLA Workshop, Socorro, NM June 2017

Wide-field Continuum Imaging (3) Baseline-dependent averaging: Nchan/dt goes as B2 Could save a factor of 10-ish in data rate, depending on distribution of dishes Even larger savings (factor 100-ish) if abandon wide-field imaging at highest spatial resolution Correlators always produce more data than can initially be handled Flexible spectral & temporal averaging is critical ngVLA Workshop, Socorro, NM June 2017

Other potentially nasty (aka driving) requirements RFI Depends critically on observing bands (at the moment) Ideal: tailor temporal/spectral res’n to expected RFI at a given frequency Nastiest known for SKA1_Low/Mid requires sub-msec resolution Linearity issues => number of bits Frequency confinement: multiple channelization stages with anti-aliasing at each stage (cf. WIDAR sub-bands & freq. offsets) Fast Radio Burst localization (low frequencies) sub-msec temporal resolution sub-arcsec spatial res’n  ~6800 channels 1.2-2.0 GHz ngVLA Workshop, Socorro, NM June 2017

Outstanding questions Do we need to make tied-array beams? How many? With what bandwidth? E.g., for pulsars & for VLBI Do we need a transient buffer? Covering how much bandwidth, for how long? Solar imaging requirements? Detailed requirements on: Spectral leakage Bandpass flatness Bandpass stability …in the past this has mostly been “do the best you can for not too much $$” -- maybe still OK? For most scientifically interesting parameters (e.g., #of zoom windows, highest spectral res’n, # of subarrays, spectral dynamic range, etc.) we can make reasonable guesses, and they’re pretty similar to the requirements for the EVLA. But there are some Big Picture questions to which it would be nice to know answers. ngVLA Workshop, Socorro, NM June 2017

Questions and Discussion? Thank you.