1. Philippe Picard1EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 EMBRACE station processing P. Picard Station de Radioastronomie de Nançay P. Renaud Station de Radioastronomie de Nançay C. Taffoureau Station de Radioastronomie de Nançay D. BenoistStation de Radioastronomie de Nançay V. MacaireSKADS funded L. MercierSKADS funded W. PauleSKADS funded

2. Philippe Picard2EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 One RF beam signal is the phased sum of RF signals from 72 one pol. Vivaldi elements (analog RF beamforming) Depending upon site configuration, for station processing one digitizer input is fed by a base element being:  one IF beam coming from one tile through an RF downconverter  one IF beam coming from a tileset of 4 combined tiles through an RF downconverter The IF beams are in the 100 to 200 MHz frequency range after downconverting Digitizer bandwidth: 100 MHz (200 Ms/s, 12 bits) sending a 2400 Mb/s data flow to each back-end digital input Westerbork site (144 tiles): 144 digital inputs for beam A (tiles) => 345.6 Gb/s 36 digital inputs for beam B (tilesets) => 86.4 Gb/s Nançay site (80 tiles): 20 digital inputs for beam A (tilesets)=> 48 Gb/s 20 digital inputs for beam B (tilesets)=> 48 Gb/s Station processing inputs RF beam A RF beam B EMBRACE tile 72 two pol. Vivaldi elements 500 to 1500 MHz Analog RF beamforming for 2 beams (one pol.) Station processing input data rate for EMBRACE

3. Philippe Picard3EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 EMBRACE digital beamforming Synthesis of a digital beam is done by a phased sum of all the digitized IF beams, each one being phase shifted by the proper value in order for the digital beam to point a sky direction. Using phase shift rather than true delays of IF beams signals is easier to implement, but works only in small bandwidth where a phase shift is equivalent to a true delay, requiring the use of bandpath filters before phase shifters Digitized IF beam A Antenna 1 Digitized IF beam A Antenna 2 Digitized IF beam A Antenna n Bandpath filter F, ΔfPhase shifter Ф1 Phase shifter Ф2 Phase shifter Фn + Bandpath filter F, Δf Digital beam F, Δf {AZ, El}. real data complex data This generic system gives us a digital beam in a Δf subband. Duplicating this architecture with the proper bandpath filters and phase shifts allow access to many digital beams of Δf bandwidth (Δf = 195 KHz in EMBRACE)

4. Philippe Picard4EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 To reduce development duration, LOFAR station Back-End is used as EMBRACE hardware platform for station processing. Base processing element is the Antenna Processor (AP). One AP computes the phased sums of two antennas, same RF beam, for 248 data objects {[|AZ 0, El 0 |,|AZ 1,El 1 |], one subband} called beamlets (2 beams in one subband) Xin Yin Xre,Yre Xim,Yim Xre,Xim Yre,Yim Xre,Xim Yre,Yim 248 x [Sre,Sim](Az 0,El 0,s) [Sre,Sim](Az 1,El 1,s) Subband filterSubband select Beamformer Antenna n beam A Antenna n+1 beam A 512 subbands Up to 248 subbands 2 antennas All the required APs outputs are then summed to deliver the station digital beamlets One station beamlet is the phased sum of all station antennas, same RF beam, for two sky directions and one subband. A station delivers 248 beamlets for each RF beam. Antenna Processor e.g. of digital beams configurations for RF beam A: 2 sky directions in 248 subbands (48.43 MHz if consecutive subbands) or …… or 8 sky directions in 62 subbands (12.11 MHz if consecutive subbands) or ….. or 496 sky directions in 1 subband (0.1953125 MHz)

5. Philippe Picard5EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 For two antennas X, Y and one AP, beamforming weights to phase sum these two antennas in one subband and for two sky directions {Az1,El1} and {Az2,El2} are set in a 4 x 4 matrix and the beamforming process for one subband becomes a simple matrix multiply APs input parameters Subband filter: this polyphase filter delivers 512 subbands (~195 KHz wide) using a fixed set of 16 K coefficients to define filter shape. Subband select map: 248 subbands must be selected, with no constraints on these subbands (no required ascending nor descending order, multiple use of same subband…). All the APs must use the same subband select map at the same time. Beamforming weights: these weights are the digital phase shifts and amplitude shifts required for each antenna to synthezise a beam in a sky direction. g 1 cosφ 1 -g 1 sinφ 1 g 2 cosφ 2 -g 2 sinφ 2 g 1 sinφ 1 g 1 cosφ 1 g 2 sinφ 2 g 2 cosφ 2 g 3 cosφ 3 -g 3 sinφ 3 g 4 cosφ 4 -g 4 sinφ 4 g 3 sinφ 3 g 3 cosφ 3 g 4 sinφ 4 g 4 cosφ 4 Sum(X,Y,{Az1,El1}) real Sum(X,Y,{Az1,El1}) im Sum(X,Y,{Az2,El2}) real Sum(X,Y,{Az2,El2}) im = X real X im Y real Y im * sss Complex samples of antennas X and Y signals in subband s Phased sum of antennas X and Y, subband s, towards sky direction {Az1,El1} Phase and gain shifts to apply to antennas X and Y, subband s, to point sky direction {Az1,El1} beamlet

6. Philippe Picard6EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 Subband width: 195.3125 Khz (input data sampled at 200 Ms/s) φ is one sky direction (Az,El) Separate Subbands Select Subbands 2 data flows X and Y (12b. real) 2 x 2400 Mb/s 512 X subbands 512 Y subbands 2 polyphase filter banks 16 K coef. Subband Select map 248 X subbands 248 Y subbands Weights 248 X weights 248 Y weights 512 x [Xre + jXim] b [Yre + jYim] b b = 0 to 511 (18b complex) 248 x [Xre + jXim] s [Yre + jYim] s s in [0, 511] (18b complex) 248 x [Sre + jSim] s,φ0 [Sre + jSim] s,φ1 s in [0, 511] (18b complex) 1024 samples time frame => 248 x [FSre + jFSim] s,φ0 [FSre + jFSim] s,φ1 s in [0, 511] (16b complex) 248 x [PSre + jPSim] s,φ0 [PSre + jPSim] s,φ1 s in [0, 511] (18b complex) Form Beams X Form Beams + Data rate => 7.2 10 9 b/s3.4875 10 9 b/s 3.1 10 9 b/s Max[nb.s] = 248 Max[nb.s x nb.φ] = 496 Max[nb.s x nb.φ] = 496 Constraints => two tiles or cells of combined tiles Station output From LCU (1s time frame) From previous AP in the chain To next AP in the chain Antenna Processor datapath

7. Philippe Picard7EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 AP0 AP1 AP2 AP3 BP CONF PHY SERDES data 11 x 800 Mb/s 2 x 800 Mb/s control X0 input 200 Ms/s 12b. Y0 input X1 input 200 Ms/s 12b. Y1 input X2 input 200 Ms/s 12b. Y2 input X3 input 200 Ms/s 12b. Y3 input RSP board 100 Mb/s raw Ethernet Monitoring and control 1 Gb/s raw Ethernet Station data (partial) From previous board 4 x 2.5 to 3.125 Gb/s (Infiniband) To next board 4 x 2.5 to 3.125 Gb/s (Infiniband) 4b. MII 8b. GMII Station processing board APs : FPGA using 90 nm process

8. Philippe Picard8EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 Select Subbands Form Beams Output Beams Compute Subbands Statistics Compute Cross Correlations Compute Beams Statistics Correct for Calibration Nulling of Interferers Calculate Initial vector for Beam forming Calculate Calibration Detect Interferer Calculate Projection Matrices for nulling Subband frequency Array geometry Subband frequency Array geometry Source coordinatesInterferers coordinates Station Control Unit Subbands to be processed Output mode Time stamp LO1 beam A LO1 beam B LO2 Data recording Post processing Local Control Unit Antenna data From RCU N Tiles 2 RF beams Store Tile array settings Calculate Tile array settings Source coordinates External Correlator Interface Separate Subbands Switch Monitoring and Control software 2N x 200 Ms/s 12b.

9. Philippe Picard9EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 EMBRACE station data output  1s averaged power of all 512 subbands for all antennas: subbands statistics  1s averaged power of all 248 station beamlets: beamlet statistics  1s averaged cross correlations of all antennas, for one subband output of up to all 248 station beamlets on up to 4 x 1 Gb ethernet links Locally stored 1s averaged data, for each RF beam: External Correlator Interface with two analog outputs  output bandwidth: up to 20 MHz  starting frequency: 0 to 40 MHz Analog output for external analyser systems: High temporal resolution data (5.12 µs), for each RF beam:

10. Philippe Picard10EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 External Correlator Interface: outputs two beams in 0-20 MHz analog signals 20 MHz bandwidth requires 103 consecutive subbands (20.14 MHz). Input of ECI: 103 beamlets of consecutive subbands and 2 sky positions (the same for each beamlet) Output of ECI: 2 x 1 data flow for the same 20 MHz bandwidth and 2 sky positions Spectral domain of output beam A (digital) Spectral domain of beam A (digital) FF+ 20.14 MHz 0 20.14 MHz Spectral domain of output beam A (analog) 0 20 MHz Digital to Analog converter + filter Digital processing Synthesis filter IF inputs of WSRT correlator (fringe stopping in IF processor) 020 180 MHz 40 Analog output for external analyser

11. Philippe Picard11EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 Half back-end RF beam A 144 inputs Half back-end RF beam B 36 inputs ECI Digital beam A(Az 0,El 0 ) in one 20 MHz bandwidth, to analog output 1 Digital beam A(Az 1,El 1 ) in the same 20 MHz bandwidth, to analog output 2 Data storage of up to 124 beamlets from RF beam A and 248 beamlets from RF beam B mode 0 mode 1 Note: Same configuration available with RF beam B to ECI and RF beam A to data storage Data storage Gb switch Half back-end RF beam A 144 inputs Half back-end RF beam B 36 inputs ECI No analog output 1 No analog output 2 Data storage of up to 248 beamlets from RF beam A Data storage of up to 248 beamlets from RF beam B Westerbork array station processing configurations 18 RSP boards for beam A and 5 RSP boards for beam B mode 0 Data storage Gb switch tiles tilesets tiles tilesets Data output configurations

12. Philippe Picard12EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 Data recording, access to post processing P. Renaud Station de Radioastronomie de Nançay J. Borsenberger Observatoire de Paris F. ViallefondObservatoire de Paris Henrik OlofssonObservatoire de Paris S. TorchinskyStation de Radioastronomie de Nançay S. PomaredeSKADS funded Ongoing work on specific computer hardware configuration using COTS components to allow real time recording of at least 124 beamlets (2 x 1 Gb ethernet link) and recording software Storage capacity to record N beamlets ( GBytes or Tbytes) beamletsdata flow 10 min 1 hour 5 hours10 hours 62 93 MB/s 54.2 325.5 1.6 3.2 124 185 MB/s 108.5 651 3.2 6.4 186 278 MB/s 162.8 976.4 4.8 9.6 248 370 MB/s 217 1.3 6.4 12.8 Real time data recording

13. Philippe Picard13EMBRACE station processing SKADS Conference, Limelette, 4-6 November 2009 Tools for acquisition software and post processing: Work on an Embrace data model in order to generate API in Python / C++ to deliver procedures to be used in test and observation software (F. Viallefond) Work to define an Embrace Measurement Set to be used in data acquisition and post processing tools. Data tools