Technology and Functionality of the EVLA Correlator (Next Corr Workshop, June 27, 2006)

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Outline EVLA project overview. Correlator capabilities. Correlator architecture. Technology. Schedule.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 EVLA Project Overview NRAO: –Upgrade VLA from the current ~100 MHz BW to 16 GHz (8 GHz/pol’n). –New receivers: 1-50 GHz contiguous frequency coverage. –Replace waveguide system with digital fiber system. –3-bit (8x2 GHz) and 8-bit (4x1 GHz) sampling. New correlator (NRC Canada).

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Correlator Capabilities 16 GHz/antenna (8 GHz/poln). 32-antenna system…expandable. Can tradeoff BW for #ants and #beams; can proc multiple narrow (<1 GHz) bands; VLBI ready. 3/8-bit ADC (at antenna), 4/7-bit re-quantization and correlation…more bits/less BW possible. All digital +/-1/32 sample delay tracking; 0.26 sec delay range. 3-level phase rotation in correlator chip: sub-sample delay, LO offset removal, earth-rotation phase, bias/birdie/aliasing de-correlation.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Correlator Capabilities 16k channels/baseline wideband; up to 4 Mchannels/baseline narrowband with “recirculation”. Up to 144 tunable multi-stage digital filters per antenna…’slot’ restrictions in 1 st stage of filter. Use filter logic for sub-band multi-beaming and/or narrowband delay. ~60 dB spectral dynamic range. “WIDAR” technique (hybrid).

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Correlator Capabilities High-speed pulsar phase binning; 2000 bins/product, min ~15 usec bin width. High-speed dumping. 1 Gbps Ethernet/board standard. Up to 10 Gbps/board. 25 Mbytes/sec to archive. All digital phased output…1 GHz…expandable to full BW. Unlimited sub-arrays. Multiple VSI I/O. Configurable for virtually any radio telescope configuration…including auto- correlation. Uses VLBI standard frequencies (256 MHz…). Can incorporate VLBA into “excess” correlator capacity.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Correlator Architecture Station Board: –6U Fiber receiver mezzanine card (NRAO). –Cross-bar switch FPGA. –Delay module mezzanine card (+/-0.5 samples delay). –Wideband autocorrelator. –Multi-stage digital filters (1 FPGA per sub-band). Cross-bar switch and sub-and multi-beaming delay memory. 4 stages…128 MHz…31.25 kHz BW out. Power measurements for sub-band stitching. Real-time RFI blanking per sub-band. CPU-settable scaling factors/re-quantization for max sensitivity/dynamic range. State counts, lag-0 power, phase-cal. Narrowband sub-sample delay with 16-step FIR interpolation. –Cross-bar switch and mux to 1 Gbps for data transport to Baseline Board.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Correlator Architecture Fanout Board: –Dual 1-8 fanout of signals (4-wafer ‘stack’, 1 sub-band, all basebands, one antenna) for baseline rack signal distribution. Baseline Board: –8x8 matrix of corr chips fed by 8 ‘X’ and 8 ’Y’ FPGAs (recirculation, phase generation, 7-bit handling, cross-bar). –Corr chip: 2048 c-lags, in c-lag “cells”. Full 4-bit multiply, no trunc,  sec int. Cells can be concatenated. –Dedicated LTA FPGA +256 Mbit RAM for each corr chip; high- speed phase binning and recirculation. –Output 1 Gbps Ethernet via FPGA. –Can set corr chip and board for autocorr mode: 64 independent 2048 channel, 128 MHz autocorrelations. –Can process up to 32 stations (1024 correlations), 128 MHz, 1 poln product, at 64 channels/baseline.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Technology Use Xilinx Virtex-IV, and Altera Stratix II/GX/Cyclone FPGAs. Standard cell correlator chip, 4 million gate, 0.13  m. PD~3.7 W 256 MHz, 1 V. Use ‘Accel’ point-of-load reg to minimize power, track chip speed with time. BGA packages throughout, some gull-wing memories/drivers.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Technology 8 different boards in total: Station Board, Baseline Board: 12U x 400 mm, 28 layer, ” trace-’n- space, 0.125” thick. Phasing Board: ~6U x 160 mm(?). Fanout Board: 6U x 100 mm, with re-sync Stratix II FPGA. Common Backplane/midplane (3U x 25 mm); signal I/O, power, ID…used everywhere…no monolithic backplanes. HM 2.0 mm straight-thru. All press- fit. Delay module (~5”x3.5”)mezzanine card: FPGA+DDR SDRAMs PC/104+ COTS CPU + PCMC card. RPMIB (very simple; diodes, optos) for rack/fan power monitor and control

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Technology 1 Gbps station-baseline data transport on high-density Meritec 2 mm hardmetric cable assemblies. Make correlator highly configurable for various correlator configurations (e.g. e- MERLIN). Use Altera’s 1 Gbps mux/demux with dynamic phase alignment…built into Stratix chips. About 500 rack-to-rack cables, 2500 intra-rack cables (<1 m) in EVLA correlator. All cables plug into 3U x 25mm Common Backplane.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Baseline Board (12Ux400 mm) PCB-back

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Baseline Board PCB-front

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Thermal vias “pocket” for corr chip heatsink

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Technology Use central -48 VDC COTS power supply, DC-DC supplies on each board. Provides battery backup (5 min req). 24, 24” racks (2.5’ x 3’ x 7’), each one holding up to 16 large boards in two crates, and Fanout Boards in 6U crate. All hot-swappable, including fans. On-line detection of communications errors, temps, voltages…remote shutdown/power cycle of any board through central PXI chassis/CPU.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Thermal mock- up test rack

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Technology Software: –Embedded PC/104+ COTS CPU on each large board. Cheap, fast, easily available, easily replaceable, many vendors. –RT Linux…”Unix-style” device drivers. –System…board…chip-level GUIs for initial testing. Useful during operations for continued debugging, low-level access. –MCCC—central host computer maps high-level requests from EVLA M&C to embedded processors, via XML messages. –CPCC—central power control computer. –Backend—array of COTS PCs connected to Baseline Boards via commercial GigE switches.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Schedule Slipped about 1.5 yrs from original 2001 schedule…various reasons. 10 proto corr chips arrived June 20/06. Baseline Board…probably August/06…some PCB fab problems have delayed delivery. Station Board…signed-off design for fab quotation…probably get assembled board late September/06. Depending on how well testing goes: –1 st prototype corrs at VLA/Jodrell Bank…mid ’07. –Full production mid Full installation early –Turn off old correlator end of 2009.

Next Gen Corr Workshop: EVLA Correlator—Carlson June 27, 2006 Summary Response Type of correlations: all Output req: spectral + continuum Special processing req: tunable sub-bands, high spec res, pulsar, 4/7-bit corr, RFI blanking/robust; RFI excision in backend CPUs eventually; possible RT RFI cancel. Input BW + digitization: 8x2GHz 3-bit; 4x1GHz 8-bit; dedicated fiber I/F on mezzanine card; Other BW/digitization supported; VSI I/O. #spec channels at max BW: 16k, increases by 2X with each decrease in BW by ½. #baselines correlated: 528 baselines, 163,840 cross-corrs Integration times: min ~15 usec with binning. 11 msec standard…scalable with backend and O/P link; unlimited max. Dynamic range: ~60 dB spectral. Scale: large, near limits of technology but not “bleading edge”. Technology: FPGAs, custom hardware, 1 ASIC, COTS CPUs/network, standard form factor. Scalability: unlimited but high flexibility not necessarily best approach for ‘large N’. 4X baselines possible with new ASIC, same architecture. Flexibility: high: BW/Nant/Nchan/Nbeam/Nbits; RT, nRT VLBI, auto, connected element, integration/dumping independent of system timing. Architecture: “WIDAR” (hybrid XF). RFI mitigation: Nbits, RT blanking, high dynamic range.