New Technologies for Future VLBI Correlators B. Carlson High Res. Astronomy, June 8-12, 2003 Socorro NM

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 2 Outline Key architecture and design issues. Review of correlator groups/developments. EVLA Correlator for VLBI. Design challenges. Design methodologies. Design tools. “Vision for the Future”

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 3 Key Issues Architecture: determined by requirements –BBC configurations: are parallel processing methods required? –Bandwidth: directly affects digital clock rate, choice of technology, cost. –Number of antennas: small…large N, may affect flexibility, signal processing choices/tradeoffs. –Number of spectral channels. Flexibility requirements determine how spectral channels are allocated to bandwidth.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 4 Key Issues Architecture cont’d: –Dynamic range: affects number of bits used at various stages. Can have a major impact on the choice of architecture. –Baseline, real-time/fiber delays, recording based: affects delay tracking hardware complexity/cost. –Flexibility: specific antennas/BBC configurations OR wide range of different antenna handling abilities? –Number of bits in correlator: can affect where digital filtering (if any) is performed (i.e. at antenna or at correlator).

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 5 Key Issues Architecture cont’d: –Delay and phase tracking at the antenna: simplifies correlator design; more complex H/W and S/W at antenna; “got one chance to get it right”; some say YES, some say NO.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 6 Key Issues H/W design issues: determined by architecture and speed requirements –Low, medium, high-speed. Transmission lines? Propagation delays on circuit boards important? Signal integrity? Power dissipation? –Station-to-baseline signal distribution: critical at high speeds, easier at medium to low speeds. May or may not be issue. –H/W or Software correlator: determined by bandwidth, number of antennas, number of channels, duty cycle. –Delay and phase tracking: dynamic range, precision requirements, drive how these are handled.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 7 Key Issues S/W design issues: determined by architecture and flexibility requirements. –“Observing modes”-based design? –Goal/science-oriented design? –Data handling performance requirements: how fast does data have to be dumped? How many channels/baseline?

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 8 Correlator Groups + Recent Developments XF Correlators –MkIV/JIVE/… (Haystack/European cons.) – 1k…4k channels/baseline; 2 bits; 64 MHz clock rate. –Specifically built for VLBI.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 9 Correlator Groups + Recent Developments FX Correlators –ATA (SETI Institute/UC Berkeley) – ~1k chans/baseline; 4 bits; 350 antennas(!) (60k baselines); 100(?) MHz clock. –ATNF research into ALMA (J. Bunton) –Japanese ALMA 2GC proposal – ~>10k(?) chans/baseline; 4 bits; 64 antennas; 125(?) MHz clock

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 10 Correlator Groups + Recent Developments FX Correlators cont’d (All) use poly-phase FIR/FFT filterbank for superior performance over previous generation (VLBA, Mitaka) FX correlators. FX design may limit flexibility in handling BBs, and assignment of channels to multiple spectral regions of interest. Rely on re-quantization to 4 bits after FFT to deal with station-to-baseline bandwidth problem…could limit dynamic range depending on how this is done. Not particularly built for VLBI, but could handle VLBI with front-end delay/phase compensation. Possibly the optimum solution for large-N correlation.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 11 Hybrid Correlators (digital filter…XF correlation) –EVLA “WIDAR” – NRC Canada – 16k…4M chans/baseline; 4/8-bit; 40+ antennas; high dynamic range (50-60 dB); 256 MHz clock –U.S. ALMA – NRAO – recently proposed to be upgraded to hybrid capability – up to 32k chans/baseline; 2/4-bit; 64 antennas; 125 MHz clock. –European ALMA 2GC proposal – ASTRON et. al. – 16k…32k(?) chans/baseline; 2/4-bit; 64 antennas; 125 MHz clock (All) perform digital filtering followed by XF correlation. Various dynamic range capabilities. Various flexibilities, “ways of doing things”. EVLA correlator is only one with VLBI capability “out of the box”…U.S. ALMA could with front-end phase tracking…European ALMA has fundamental ability if so chosen… Correlator Groups + Recent Developments

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 12 Correlator Groups + Recent Developments Software Correlators –Australia (S. Tingay et. al.) – Swinburne University of Technology parallel supercomputer: 180, 2 GHz PCs. (XF?); a few antennas; modest bandwidth; low duty cycle. –NRAO (T. Morgan) – feasibility study using the Cray X1 supercomputer. May have advantages over PC cluster S/W correlators because of “matrix math unit” that allows bit-level operations. FX. Too expensive for EVLA, but for smaller-scale VLBI correlator…? –LOFAR – ASTRON/NRL/Haystack – FFT (spatial) beamforming or FX correlator?. 32 MHz. Mixture of H/W and S/W elements?

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 13 EVLA Correlator for VLBI—Overview Bandwidth, flexibility, and signal processing method makes it suitable/useable for VLBI. Uses VLBI-standard sample rates. Can handle antennas with a mixture of bandwidths w/ or w/o delay/phase tracking at antennas. Recording-based or real-time VLBI (fiber/VSI interfaces).

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 14 EVLA Correlator for VLBI—Overview Sub-band multi-beaming for multi-source observing/in-beam phase referencing. Tradeoff bandwidth for number of antennas/beams w/o rewiring the correlator. 16k chans/baseline…4M chans/baseline w/ recirculation. 1, 2, 3, 4, 8-bit capability. High-speed, 2x1000 phase bins Digital phased-EVLA output.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 15 EVLA Correlator—Station Brd Processing Flexible BBC handling: 2 2 GHz each…32 BBCs <=128 MHz each second delay buffer using inexpensive SDRAM…complex design. All-digital +/- 1/16 th sample delay tracking (wideband…narrowband) 32 independent digital filters…bandwidth, beam, location in BB. Probably implement in gate array. Phase-cal extraction, quantizer statistics. 2 Pulsar timers for gating/binning. Data path switching. VSI input for connection to new-generation VLBI recorders. VSI output for recording sub-band data (EVLA Phase-II reqs)…could use board as filterbank board at antenna if desired.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 16 EVLA Correlator—Baseline Brd Processing 64, 2048 complex-lag correlator chips: lags/cell; fringe- stopping; high performance readout for recirculation + phase binning. –Corr. chip design complete: 4 Mgates; est 2.7 W PD; est $400k NRE, $90/chip in 10k qty, $50/chip in 25k qty. Recirculation by up to a factor of 256 for up to 256k chans/correlation (4M chans/baseline). High performance Long-Term Accumulator (LTA). Output UDP/IP frames containing data…for switching to backend.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 17 EVLA Correlator Configurations NRC will deliver a 32-station correlator (16 GHz/station) –For EVLA Phase-I this leaves 5 “spares”: 5 16 GHz; 10 stations at 4 GHz (4k chans/baseline); 20 1 GHz (1k chans/baseline). –This extra capacity could be used for the VLBA (+some control S/W). EVLA Phase-II –NRAO will upgrade to 40 stations. –27 EVLA + 10 NMA…leaves 3 spares (3 stations…6 stations…12 stations)

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 18

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 19 Design Challenges Speed –Most if not all signals require transmission lines. –Power dissipation is everything. can affect choice of technology: FPGA or ASIC + feature size. can limit functionality: “what can be put on a chip”. Complexity –Astronomers want “everything”. –Requires increasingly complex H/W functions for performance. Moore’s Law yeah, but with increasing speed and complexity, everything gets more difficult for engineers.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 20

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 21

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 22 Design Methodologies To cope with increasing speed and complexity: –HDL/RTL design for chip logic (“code that describes hardware”) Incremental code and test…hierarchical design. Much faster than schematic-based design. Code can be written in technology-independent fashion to target the “latest and greatest” chips. Code can be re-used… “Testbenches” for testing can take as long to write as the logic function itself! –Signal integrity and timing analysis through the development cycle. –High performance ($$), constraint-based PAR (Place And Route) tools both for chips and circuit boards. –Post-PAR simulation, to verify signal integrity and timing margins…pretty darn sure its going to work before PCB fabricated.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 23 Design Tools We are using “Mentor Graphics” –FPGA Advantage: complete suite of tools for chip HDL+graphical hierarchical design, simulation, and “synthesis”. –“Expedition” PCB tools: “Signal Vision” for transmission line “what-if” analysis. Schematic capture…constraints entry. PCB constraint-based PAR: unbelievable capability. Post-PAR signal integrity analysis, timing margin analysis. Thermal analysis. –Cost: about $250k for the EVLA correlator team; ~$40k/yr maintenance…already finding it is well worth the cost.

B. Carlson, 2003-Jun 8-12 Socorro NM High Res. Astronomy - New Technologies for Future VLBI Correlators 24 “Vision for the Future” Maximum bandwidth possible. Moderate size N. ~10k or so channels/baseline. Disk-based and eventually fiber-connected VLBI. Flexibility, able to “absorb” outputs from variety of antennas (GBT? DSN? EVN? LAR?) Delay and phase tracking at the correlator (my choice, my guess). EVLA and VLBA merge into one instrument, into one correlator. EVLA correlator suited to meet all of these requirements for the foreseeable future…