Challenges in Hybrid DSP/FPGA Implementations of Optimal Beamforming Bogdan Vacaliuc, Sundance DSP, Inc. 1. Sundance is a TI 3rd Party; Should get an approval from MathWorks about a ‘Connections Program’ soon and would like to become a Xilinx Xpert Member

Overview Company Highlights and Background BF1 System Requirements Top Level Design Partitioning Case Studies Current Status Future Expansion Options

Sundance Established 1989, Privately held Company; Assets in excess of $2M Sundance Design, Manufacturing, Test in England and USA; Sales Offices around the World Sundance is ISO9100-2000 compliant company and has been since 1998. We have own offices in Italy, Genova One is a PhD in software and does ‘System Development/Proposal’ for Southern Europe Other is ‘MarCom’; Sales and general admin We have own sales office in Reno, Nevada, USA Location is cost-driven!! One is a PhD in software; two are ‘Electronics’ engineer and they offer ‘System Development/Proposal’ for America Two other does ‘MarCom’; Sales and general admin, We have our own ‘Rep’ in France and in Canada We sell to rest of World either directly or through distributors. The England office does all the R&D, design, manufacturing (has own BGA facilities, with inspections systems, etc.) and test + admin and Web + Sales to Northern Europe. Location is in an old ‘Toys’ factory; Very appropriate for a company like Sundance. Currently we have 5x VHDL persons; One DSP H/W person; One DAQ H/W person and 3 software persons in R&D Three ‘Test’ and ‘System’ Engineers – Do testing of all Modules and integration of development systems. Three ‘Production’ Engineers – and we build approx. 3000 board/year. Three in ‘General administrations’ Two in ‘IT’ – That is just to keep the ‘Spam’ and ‘Virus’ away! - and Me!! – says the Donkey! Sundance has good working relationships with ‘Ex-Sundance’ people that has left to start own ventures; We currently have resource-pool of 10-12 people that has during the last 5-6 years done specific developments for Sundance or more typical, for our clients. The turn-over was close to $4mill in 2002.. – down 25% from the highest point, but we still the same people and we are seeing some signs of growth for Year 2003.

World Leading Users Michigan Tech University SPAWAR Systems Center - Charleston Lockheed Martin Cymer NASA Raytheon TRW General Dynamics Philips Medical Motorola L3-Communication Lucent Technologies MIT Rolls Royce ... We typical have 25 new customers per year; - and we lose approx. 25 customer per year!! – The current list of ‘Active’ customer is approx. 80.

Concept of Modular Design Module Carriers PCI cPCI VME PMC Modules A/D,D/A,I/O DSP,FPGA IMAGING,MEMORY Systems Data Acquisition Medical Industrial Control & Monitoring Carriers can be SMT300 = 3U PXI/CompactPCI; One Module site SMT300Q = 6U CompactPCI; Four Module sites SMT310 = Short length PCI; One Module site SMT310Q = full length PCI; Four Module sites Modules More than 100 different combinations!! Systems SMT7xxx are ‘Development Systems’ with a number of DSPs or FPGAs – None specific applications SMT8xxx are ‘Application Systems’ – custom specific Some are for Medical, like SMT8056 Some are for Control, like SMT8025 Some are for DAQ, like SMT8036/SMT8046

Processing modules SMT374-300 SMT361Q SMT398 SMT318-SX55 Dual 300 MHz TI C6713 Xilinx Virtex II XC2V2000-4 256MB SDRAM 920MB/s I/O bandwidth Quad TI C6416 Xilinx Virtex II XC2V2000-4 4MB internal memory 920MB/s I/O bandwidth Xilinx Virtex II XC2V8000-4 4MB ZBT SRAM 2MB QDR SRAM 1.6GB/s I/O bandwidth Dual Xilinx XC4VSX55-12 1024 XtremeDSP 1.6GB/s inter-FPGA I/O 2.5GB/s I/O bandwidth

Platforms PCI SMT310Q cPCI SMT300Q Embedded SMT180 VME SMT328

Company Highlights and Background BF1 System Requirements Top Level Design Partitioning Case Studies Current Status Future Expansion Options

BF1 System Requirements Digital System for processing 8 element ULA or UCA (receiver only) Target Signal: Family Radio Service (FRS) 462.5625MHz to 467.7125MHz 25KHz channel separation Intermediate Frequency 21.4MHz Center 22.5MHz Bandwidth Able to separate “talkers” on the same frequency USB 2.0 Interface to HOST 40W power specification is key design requirement

BF1 System Requirements (cont.) Flexibility in implementing different beamforming algorithms Flexibility in implementing different channel (de)modulation algorithms Multi-Channel operation required The more channels, the better Channel refers to

Company Highlights and Background BF1 System Requirements Top Level Design Partitioning Case Studies Current Status Future Expansion Options

Top Level Design A tuner for each antenna An ADC for each tuner Channelizer Beamformer Demodulator A tuner for each antenna An ADC for each tuner One channelizer A beamformer/demodulator for each channel Output to Host

ADC Selection Fs > 75MHz Ideal Fs is 102.4MHz Fs > 2*(IFcenter+(IFspan/2)+Guard) Ideal Fs is 102.4MHz Fs = Fc*M M= # FFT points Fc = 25KHz M is > 3000 for Fs > 75MHz Pick 4096 point FFT, Fs = 102.4MHz Maximize #ADC per module Maximize ADC resolution Pick SMT364 Quad 105MSPS ADC 14-bit resolution 40W power specification is key design requirement

CLOCK Selection Need to synchronize 8 ADCs (or more) On board clock is not synchronized between all ADCs Each pair of ADCs are clocked together SMT364 requires two external clocks per module For beamforming it is essential to have high stability clock sources that do not drift over time MOST IMPORTANT component 40W power specification is key design requirement

CLOCK Selection (cont.) Mfg. Part Number Jitter Metric[1] Peak SNR[2] Greenray YH1441-B17-3.3-102.4MHz 0.041ps 91.6dB Raltron OX6551A-LX-3-102.400 0.031ps 94dB SBtron SBOC25BBS-3.3V-Sine-102.4MHz 0.020ps 97.8dB Valpey-Fisher VFTCS-B58L3S-102.4MHz 0.005ps 109dB Vectron C4530-D107-SV033-RFS-B1-102.4MHz 0.875ps 65dB Evaluate oscillator performance by considering phase noise tables Estimate RMS jitter and compute absolute maximum SNR Make sure maximum SNR >> ADC specification That way, if you want 2 DSPs w/512MB each you can have that, and the other two are powered off. Each of those will be a different FPGA bitstream. You can load them all in the FLASH and have the bootstrap code decide which one is appropriate. [note: the DSP0 bootstrap code loads the FPGA bitstream from flash]. NOTE: C6713 limits memory addressability to 512MB, due to address line limit (pg. 17 datasheet) [1] Computed from the on-line calculator at: http://www.raltron.com/cust/tools/osc.asp [2] SNR = 20log10(1/(2*PI*Fsignal*Tjitter)) as described in: http://www.analog.com/en/content/0,2886,760%255F788%255F91502,00.html

SMT399-F102.4 Module Fixed Frequency Clock Source External Clock (build option) 4-way Power Splitter < 3° phase variance Amplifier with phase adjustment MMBX or SMA (build option) On-board linear regulator and power filter MMCX (build option) Flexible power input (TIM or EXT) Fine Frequency adj. for calibration Fixed Frequency Clock Source Frequency Stability: 50ppb Aging: 300ppb/year Option for external clock input

System Level – Analog Connections

ADC Correction and Normalization Parameters (offset, gain, delay) Offset For BF1 System each ADC is channelized independently, so offset not a problem For systems that interleave ADC to increase the effective sampling rate it is critical Gain Can be adjusted by ADC parameter Can be adjusted numerically Numerical adjustment is easier Delay ADCs can start at slightly different clock edges (even with a trigger pulse distribution) Wang pointed out that feedback loop needed in their system

Design Level – ADC block detail FC201 for ADC channel correction FC202 for quadrature conversion Common handling (in FPGA firmware) of all channels Prepares data stream for channelization

Company Highlights and Background BF1 System Requirements Top Level Design Channelization Case Studies Current Status Future Expansion Options

Channelization Polyphase Filter In the BF1 System Each channel represents a frequency band M is chosen with respect to Fs (102.4MHz) and channel spacing (25KHz) In the BF1 System Fs is 102.4MHz M is 4096 Provides 4096 channels @25KSPS DDC ok when you know your channel

Channel Partitioning and Distribution Bandwidth problem 8 * 4 * 102.4MHz is 3.2GB/sec No module has that amount of I/O capacity Separate channels based on region of interest. Beamforming is done on each channel separately Combine output from all ADCs On different FPGAs in our system Wang pointed out that feedback loop needed in their system

Channel Partitioning (cont.) Use an FC108 block for each ADC input FC108-D is a double-data rate version of FC108 Fits onto XC4VSX55 220MHz with -12 part FC203A exchanges high/low channels and ADC streams FC203B formats the selected channels into a multiplexed “frequency domain highway” NOTE: still using the old numbers

FDMA Highway Enables distribution of multiple channels from all ADCs to other modules 40us frame time represents the channel spacing (25KHz) (FC202 decimated Fs by 2 with no loss of spectral information)

Putting it all together…

Software and Firmware Development -Simulink/Matlab -C Reference Models Modeling -PARS -System Generator -Diamond + CCS + ISE Code & Build DSP -Debugger,Simulator -Data I/O, Scripting -Hardware In The Loop Debug -Real Time Analysis -Profiling (CCS) -Timing (GPIO) Analyze & Tune

Develop System Model Tasks & Channels Design your application as a number of communicating tasks.

Generate System Firmware Map onto hardware Then place those tasks on processors.

SMT319 with Xilinx Virtex II XC2V2000-4 Software MATHWORKSTM SimulinkTM System Generator PARS Level 2 Level 1 Sundance Target Hardware Code Composer StudioTM Traditional VHDL Development Tools Sundance has developed two products for the integration of the MathWorks software with the Sundance hardware SMT6040 Provide ‘Blocks’ to control our FPGA Modules Typical systems application and examples are included. SMT6050 Provide ‘Blocks’ for all the different DSP Modules System specific application that allow interface to the DAQ Modules This setup requires: Matlab Software Simulink Software Xilinx Software TI Software Sundance software - and has a total cost close to USD30k!!! - but can easy save man-years of effort - allow upgrade to faster hardware ?????? SMT319 with Xilinx Virtex II XC2V2000-4

Photo of Initial Prototype System

? Roadmap System-System Interconnects PARS Enhancements Expand on number of ADC channels Enable additional antennas PARS Enhancements Use HDL Coder to enable Simulink->FPGA Increasingly automatic code generation Sundance will lead in modular, deployable signal processing Going to leave you with a big question mark…

Thank You Or… The combination of Sundance broad range of hardware building blocks for creating …. Sundance – a Partner of MathWorks, Texas Instruments and Xilinx, provide….. Every development system will require flexibility as offered by Xilinx’s FPGA, easy programmability as found with TI DSPs, Modular hardware platform is provided by Sundance and the combination of Matlab <> Simulink Tools provide rapid development time and portability.