The ALMA Correlator Gianni Comoretto, J.C. Webber, A. Baudry, C.M. Broadwell, R. P. Escoffier, J.H. Greenberg, R.R. Treacy, P. Cais, B Quertier, P. Camino, A. Bos, W Gunst, Workshop on New Generation Correlators Groningen, 27th-29th June, 2006

Contents The ALMA interferometer The correlator Time and frequency division modes Characteristics and operating modes First test results Summary

The Atacama Large Millimeter Array Interferometer Joined NRAO-Canada-Eso-Japan project 64 elements connected interferometer 5000 m altitude site, 1.5 mm precipitable H 2 O Total collecting area 7000 m 2 Baseline spacing => 12 Km: 200 GHz GHz receivers in 10 frequency windows Compact array with smaller antennas 12m dish at VLA test site Chainantor Site

The Atacama Large Millimeter Array Interferometer 8 GHz total bandwidth – dual polarization 4 independently tunable IF channels Fringe stopping at 2 nd LO 3 bit sampler, 2-4 GHz band Variable clock phase for fine delay compensation Digital fiber data transmission: 3 bit x 8 channels x 4 GS/s on multicarrier fiber Minimum dump time: –1ms for autocorrelation data –16 ms for visibility data in continuum mode –500 ms for full spectroscopic modes (128M visibility points, 1GB/s data rate) Low resolution spectroscopic mode needed for continuum observations to remove spectral lines Dual sampler

The ALMA correlator 4 quadrants, each one processing single 2 GHz IF channel 4 polarization products, 64 antennas Correlator chip: 4K lags at 125 MHz clock, 2 bit sampling Correlation card: 32x32 antennas, 4 polarizations, 125 MHz clock 4 cards process 1/32 of the IF bandwidth: 512 cards total Time division and frequency division modes One quadrant Station rackCorrelator rack power supply Computer

System design evolution Initial design: pure lag correlator, with band selecting digital filter –Time division architecture: each correlator plane analyzes a time segment of the input data –Limited frequency resolution at full BW: 64 points/IF (31 MHz) –Full resolution (2K points) at 1/32 BW Retrofitted with digital filterboard, to implement a frequency division FXF architecture –Filterbank performs 32 point frequency division –Each correlator plane analyzes one frequency segment –Resolution increases by up to 32 (1 MHz at full BW) –Filterboard mechanically, electrically and cost compatible with old digital filter Full BW time division mode mantained for “continuum” observations

Correlator structure Antenna based design Antenna unit: delay compensation, filtering, recirculating memory Correlator unit: 64x64x4 correlators, Long Term Accumulator Minimum number of inteconnections: –Each antenna unit sends 2 pol. data to 32 correlator units –Each correlator unit receives data from 64 antenna units –Point-to-point interconnections, without data duplication –LVDS interconnections with 250 MS/s data rate

Correlator structure Alma Correlator block Diagram

The ALMA correlator chip Full custom ASIC at 125 MHz clock Conventional lag type real correlator 2 bit full multiplication table. 4 blocks can be joined for 4 bit operation Individual block: 2 polarization per antenna –4x64 lags at full polarization (HH, HV, VH, VV) –2x128 lags at dual polarization (HH, VV) –256 lags with one polarization No cascading of blocks. Larger delays implemented using station based buffer memory 20 bit integration and 16 bit secondary storage

Station (Memory) Card Antenna based, dual polarization Implements: –bulk (static) geometric delay, –correlation delay, to synthetize large lags without inter-block connections –Time multiplexing modes Circular memory buffer: 1 ms being written while previous ms being read Write buffer free running, value stored at beginning of each ms 2 read buffers for each correlator plane. Different planes can correlate different data, different time interval, and/or different delay range, changing only read pointer offset

Time division mode 1 ms data block divided into (up to) 32 shorter blocks Each block sent at 125 MS/s to one correlator plane At 2 GHz BW, 32 correlator planes needed: 64 spectral points total (4 pol. mode) At reduced BW, more planes available: each plane processed data with different delay offset => more lags No interconnections between planes, all “routing” performed in memory board

Frequency division mode Digital filter to adapt input band to correlator band Each correlator plane analyzes 1/32 of the input band More correlator planes may analyze one slice (subchannel) with increased resolution (zoom modes) Each subchannel can be arbitrarily positioned in the input band Different band/resolutions possible on different (or the same) portions of the band Huge increase in correlator output: minumum integration time ~0.5 s For fast mapping in continuum mode (16 ms dump time), original time division mode retained

Subchannel stitching Stitching together many spectra with few points: Edge effects important Real correlator:  -> 0 at ->0 and -> max Shift of spectral point barycentre: error on gradients and line position Shifting in frequency by ½ channel improves phase response near edges Need to delete edge points : overlap of sub-channels required Polyphase filters not easily usable Real Imaginary

Tunable Filter Bank Array of digital receivers Each receiver selects and downconverts a 62.5 MHz frequency slot in the input 2 GHz band Delay compensation for short term geometric delay variations Implemented using 16 FPGAs (Altera Stratix2 EP2S30) Prone to single-event upset in configuration RAM: internal CRC monitor to detect configuration faults

Tunable Filter - Block diagram Configurable parameters: bandshape (via 2 nd stage filter taps, only 2 sets used), central frequency, scale factor (gain) before final requantization Mode flags: bypass mode, ½ band mode, 4 bit mode, oversampling

Bandpass filter 2 stage FIR filter: –decimating 1/32 band, 8 bit tap coefs –Half band bandshaping filter, 9 bit tap coefs > 55 dB stopband Average stopband > 60dB 0.3 dB passband ripple 93% useful bandwidth

Dynamic range and noise Dynamic range –Polyphase digital LO/mixer using 6 bit lookup tables: 0.9% loss, > 52 dB SFDR –Filter: 55 dB stopband rejection Excess noise: –3 bit + 2 bit quantizations increase quant. noise –Possibility of trading BW for sensitivity

Modes for increased sensitivity Normal 2 bit Double Nyquist: +6% Requires 2 filters Output processed by 2 correlator planes In ½ bandwidth mode, single stream at 125MS/s processed by 1 corr. plane 4 bit: +13% Requires 2 filters Output processed in 4 correlator planes

Implementation Signal distribution direct point-to-point LVCMOS Connections only between adjacent chips Interboard connections point-to-point PECL Signal integrity checkers on each data path. Programmable signal phase Clock distribution using internal FPGA PLL. Inter-rack connections 250 MHz LVDS cables Power distribution using onboard DC- DC converter, 48V bus Digital filterboard

First tests with simulated data Pseudo random noise + 2 sine lines + their harmonics due to finite quantization 10 second integration Line strength 15 dB >40 dB spurious free dynamic range Good alignment of subchannels (< 0.1%) despite different power level Problems Thermal: 100C Tj, better cooling schemes needed Mechanical: stiffners or higher (2mm) board

Summary Connected element interferometer, using digital fiber link interconnection Spectral line output, full polarization (4 pol channels) Total BW 8+8 GHz, digitized as 4+4 channels 2 GHz, 3 bit Resolution channelsmode over 2 GHz IF (single quadrant): –31 MHz 64 fast continuum mode, full polarization –1 MHz2048 Frequency division mode, full band & polarization –250 KHz 8192Full band, single polarization –3.8 KHz8192minimum band (31 MHz/IF), single polarization 64 antennas: 64 ACF, 2K CCF, with up to 1K frequency and polarization channels per baseline, total of up to 128M lags Dump time, limited by total data rate (1 GB/s): 16 ms / 500 ms. Typ. up to minutes Dynamic range: ~50 dB Technology: full custom ASIC for correlator, FPGA for filterbank Correlator board size limits n. of antennas. All ACF and CCF on one board group. Scalable in bandwidth and resolution, adding “planes” FXF with first F stage performed using tunable filterbank