1. ngVLA Correlator Drivers & Requirements
Sean Dougherty (for Michael Rupen)
NRC Canada
June 26, 2017

2. Correlator cost drivers
SKA CSP MID.CBF Team Meeting, 28th February 2017

3. The Basics: Correlator Size & Cost
A correlator…
channelizes: “F-part”
cross-correlates: “X-part”
To first order:
F-part size goes as Ndish * BW * Nbeams
X-part size goes as Ndish2 * BW * Nbeams
Correlator size is a derived quantity
Ndish -- How much collecting area can you afford, and how big are your individual dishes?
BW -- Sensitivity trade-offs for narrow/wideband feeds/receivers
Nbeams -- Sensitivity vs. FoV trade-offs for single vs. multi-pixel feeds
As will be shown, even big correlators are (relatively) cheap
F-part= channelizer – scales with # of dishes * bandwidth * number of beams (if multi-beam feeds)
X-part= cross-corr’n – scales as # of baselines= # of dishes squared, * bandwidth * # of beams (if multi-beam feeds)
FoV= Field-of-view
ngVLA Workshop, Socorro, NM June 2017

4. Scientific Requirements: Correlator Size
Number of dishes
Driven by sensitivity (ND2), FoV (1/D), uv-coverage (N2)
Probably set by total budget (dominated by dish + electronics cost) 
Maximum total bandwidth
Continuum sensitivity
Line searches & surveys
Observing efficiency (get all available information as fast as possible)
Ideally limited by receivers & samplers, not by the correlator
…but correlator cost does enter in, at the widest bandwidths
 GHz
Total BW is set by bandwidth ratio of low-noise feeds (1.85:1)
…cf. SKA1_Mid split of Band 5 ( GHz= 9.2 GHz) into Band 5a ( GHz) & 5b ( GHz= 7.05 GHz)
…ALMA Band 2+3= GHz (1.73:1)= 49 GHz
ngVLA Workshop, Socorro, NM June 2017

5. Scientific Requirements: Correlator Size (2)
Number of bems
Sensitivity vs. field-of-view: so far, hard to make multi-beam feeds competitive
SKA1_Mid deemed more effective wide-field instrument than SKA1_Survey
Flexible correlators e.g. SKA1Mid.CBF can trade BW for Nbeams
 Nbeams= 1
ngVLA Workshop, Socorro, NM June 2017

6. Latest Big Correlator: SKA1_Mid
SKA1_Mid Correlator: 197 dishes, 5 GHz/pol’n, 1 beam
Up to ~390k channels (~13 kHz over 5 GHz)
Lots of zoom options
Robust to RFI: 9+9b correlation, coarse channelization (200 MHz), sample clock offset scheme to avoid correlation of aliased signals
FPGA-based (14nm Intel Stratix-10), DDR4 memory, 26G fiber
Bottom-up costing based on detailed PBS
Vendor quotes for all key items
FPGA pricing based on 2021 buy
Scaled to 2016 €
~€17M NRE + €3M HW (F-part) + €8M HW (X-part) inc. cont.
~65 kW (F-part) kW (X-part)
Best comparison is to SKA1_Mid
Includes ~15% TMT-style contingency
Note X-part dominates cost/power for this and larger correlators (N^2)
ngVLA Workshop, Socorro, NM June 2017

7. Scaling to ngVLA: 2021 costing, 2016 €
If we were building the ngVLA correlator today:
# Antennas
Bandwidth/pol’n
Nbeams
Cost
Power
200
5 GHz 1
~ €28M
~ 240 kW
150
20 GHz
~ €45M
~ 590 kW
~ €60M
~ 960 kW
50 GHz
~€130M
~2400 kW
300
~€105M
~2000 kW
400
~€165M
~3300 kW
~€400M
~8300 kW
Mid.CBF
This is what it would cost if you were to go to production *today*. But that’s not the case…
But we are not building the ngVLA today…..
ngVLA Workshop, Socorro, NM June 2017

8. Technology marches on Difficult to forecast!
Current indicators say emphasis is on power reduction for 10nm, which maybe appears in 2020
Maybe also for 7nm?, which maybe appears in 2025??
After that ???
10nm: HW cost x2/3, Power x ½ ?
7nm: HW cost x1/2, Power x 1/3 ???
10nm: ~ €29M+ €17M (200 dishes), kW ?
€101M+ €17M (400 dishes), 1660 kW ?
 7nm: ~ €22M + €17M (200 dishes), kW ?
€76M + €17M (400 dishes), kW ?
Very uncertain!
Guesstimates from Mike Pleasance & Brent Carlson – definitely WAGs (Wild Ass Guesses) but gives some idea of how the FPGA industry is going
ngVLA Workshop, Socorro, NM June 2017

9. Scaling to ngVLA: 2016 €, 2020?/2025?? tech
# Antennas
Bandwidth/pol’n
Nbeams
Cost
Power
200
5 GHz 1
~ €28M
~ 240 kW
20 GHz
~ €60M
~ 960 kW
10nm
~ €45M ?
~ 480 kW ?
7nm
~ €39M ??
~ 320 kW ??
50 GHz
~€130M
~2400 kW
~ €87M ?
~1200 kW ?
~ €65M ??
~ 800 kW ??
400
~€165M
~3300 kW
~ €120M ?
~1660 kW ?
~ € 93M ??
~1110 kW ??
~€400M
~8300 kW
~ €270M ?
~4200 kW ?
~ €200M ??
~2800 kW ??
Mid.CBF
Even 400 dishes & 50 GHz BW is “only” 200M out of 1.5 billion, using 7nm tech -- <15%
Power usage is more of a concern
Again this assumes no really new technology, and no clever ideas – just do what we’ve designed, using cheaper parts.
ngVLA Workshop, Socorro, NM June 2017

10. Correlator Costs: Conclusions
Big uncertainties
Correlators are getting cheaper
200-dish, 20 GHz ngVLA correlator would be at most the same fraction of the budget as for SKA1-Mid (~30/325 ~10%)
400-dish, 50 GHz ngVLA correlator maybe 15% of the total
Multi-beam, wide BW feeds could change this!
Seems silly to cut corners on the heart of the telescope
Considerations to pay attention to:
ASICs – cost more up front but save power. BUT always a couple technology nodes behind…
GPUs – not yet, but keep an eye on them
N.B. New technologies and approaches will only make correlators cheaper
ngVLA Workshop, Socorro, NM June 2017

11. Second-order requirements
So far we’ve talked about the sheer size & cost of the ngVLA correlator.
There are many other requirements on e.g. the spectral resolution and dump time.
Here we consider a couple key requirements in that realm.
Second-order requirements
ngVLA Workshop, Socorro, NM June 2017

12. Wide-field Continuum Imaging – a demanding use case
Beam smearing (chromatic aberration):
dfreq/freq= (D/Bmax) * (1/K)
Perley 2004: K=4: distortion-free imaging of primary beam (10% 1st null)
Time-averaging losses: dt < Kt * (D/Bmax)
Perley 2004: Kt= 3440 (10% drop at 1st null)
vs. SKA1_Mid: Kt= 1200 (2% drop at 1st null)
D= 18m, Bmax= 300km
dfreq/freq= 1.5e-5 * (4/K) = 4.5 km/s
 dt~ 0.2 sec
Wide-field mapping is one of the most demanding experiments.
Following EVLA Memo 64 (Perley 2004), we require the following.
Note that the chromatic aberration reqt corresponds to constant velocity resolution over the band 
…so if you want, say, 1 km/s for a spectral line search over the full band, just multiply the numbers below by 4.5 .
ngVLA Workshop, Socorro, NM June 2017

13. Wide-field Continuum Imaging (2)
K=4, Kt= 3440, D= 18m, Bmax= 300km
Ndish= 200
Npol = 4 (full pol’n products)
8 B/vis (as JVLA)
Nvis= (Ndish * (Ndish+1)/2) * Npol * Nchan
= 80,400 * Nchan
Band
Nchan
Nvis
Data rate
GHz
73,242
5.9e9
235 GB/s
GHz
GHz
119,452
9.6e9
384 GB/s
GHz
33,674
2.7e9
108 GB/s
Data rates are scary! As is pretty much always true when correlators first come on-line.
ngVLA Workshop, Socorro, NM June 2017

14. Wide-field Continuum Imaging (3)
Baseline-dependent averaging: Nchan/dt goes as B2
Could save a factor of 10-ish in data rate, depending on distribution of dishes
Even larger savings (factor 100-ish) if abandon wide-field imaging at highest spatial resolution
Correlators always produce more data than can initially be handled
Flexible spectral & temporal averaging is critical
ngVLA Workshop, Socorro, NM June 2017

15. Other potentially nasty (aka driving) requirements
RFI
Depends critically on observing bands (at the moment)
Ideal: tailor temporal/spectral res’n to expected RFI at a given frequency
Nastiest known for SKA1_Low/Mid requires sub-msec resolution
Linearity issues => number of bits
Frequency confinement: multiple channelization stages with anti-aliasing at each stage (cf. WIDAR sub-bands & freq. offsets)
Fast Radio Burst localization (low frequencies)
sub-msec temporal resolution
sub-arcsec spatial res’n  ~6800 channels GHz
ngVLA Workshop, Socorro, NM June 2017

16. Outstanding questions
Do we need to make tied-array beams?
How many? With what bandwidth?
E.g., for pulsars & for VLBI
Do we need a transient buffer? Covering how much bandwidth, for how long?
Solar imaging requirements?
Detailed requirements on:
Spectral leakage
Bandpass flatness
Bandpass stability
…in the past this has mostly been “do the best you can for not too much $$” -- maybe still OK?
For most scientifically interesting parameters (e.g., #of zoom windows, highest spectral res’n, # of subarrays, spectral dynamic range, etc.) we can make reasonable guesses, and they’re pretty similar to the requirements for the EVLA.
But there are some Big Picture questions to which it would be nice to know answers.
ngVLA Workshop, Socorro, NM June 2017

17. Questions and Discussion?
Thank you.