Mark 5C VLBI Data System Alan Whitney MIT Haystack Observatory 16 June 2008 e-VLBI workshop Shanghai, China

Mark 5C Specification Mark 5C specification developed jointly by Haystack & NRAO; distributed to community in early 2007 for comment –4096 Mbps max data rate to two standard Mark 5 disk modules –10GigE data interface –Basically a dumb packet recorder (accepts OSI Layer 2 or higher) –Data source responsible for data formatting –Spec available as Mark 5 memo #57/VLBA memo #12 Mark 5C data-format spec was also been distributed to community –Supports an arbitrary # of frequency channels; single freq-chan/packet (i.e. not constrained to 2**n freq chans, unlike Mark 5A/B) –Spec available as Mark 5 memo #58/VLBA memo #13 Conduant developing new 10GigE daughter board for Amazon StreamStor card (no separate I/O board – unlike Mark 5A/B/B+) Record-only through SS 10GigE interface Playback of Mark 5C will be through host computer accessed as standard Linux files using FUSE –Natural for software correlators

Generalized 10GigE Data Distribution Concept Channelized Ethernet packets dBBC/ DBE/ VDBE

Mark 5C Development Progression Progressive versions of hardware/software will provide increasing capabilities –Version 1 requires only standard PC with Ethernet NIC and Linux file storage –Version 2 requires only StreamStor Amazon card w/o 10GigE –Version 3 requires StreamStor Amazon and 10GigE daughterboard All versions will maintain functionality of previous versions

5 Mark 5C Version 1 (aka “Mark 5c-”) Record path –Receive data from a standard NIC –Store data to COTS storage Playback path –Read data from COTS storage –Outputs data via standard NIC Provides the framework of application Limited performance capabilities < 1Gbps Interface for command / control Command line

6 Mark 5C Version 2 Added capabilities –Record path stores data to Conduant disk module –Playback path reads data from Conduant disk module Interface for command / control –Full command line capabilities for Mark5C- –Initial version of Graphical User Interface

7 Mark 5C Version 3 Added capabilities –Support for Conduant’s Hardware - 10Gbps daughter card Software Design Kit (SDK) –Data received / stored by Conduant controller card –Initialization of controller card –Packet acceptance criteria –Status of hardware / receive process

Mark 5C Status Mark 5C Version 1 software projected end June 2008 Mark 5C Version 2 software projected end Aug 2008 Mark 5C Version 3 software projected late 2008 Prototype hardware 10GigE card in checkout at Conduant Fully function prototype Mark 5C expected ~late 2008 Project Mark5C general availability Spring 2009

Mark 5 Upgrade Costs Target Existing Mk5AMk5B (requires VSI-H data source) Mk5B+Mk5C (not yet available; rough estimates) 0Unavailable$20.8K~$22.3K$20-25K Mk5A-~$3.5K (Mk5B I/O) ~$13K (Amazon plus Mk5B I/O) Est. $12-14K (Amazon plus 10GigE DB) Mk5B--~$9.6K (Amazon) Est. $12-14K (Amazon plus 10GigE DB) Mk5B+-- - Est. $2-4K (10GigE DB)

Mark 5 SATA Disk Support Problem found with SATA module backplane design with Amazon –Improper terminations prevented operation at higher clock speeds –Problem has been fixed (Board Rev 5) –Updated backplanes available without charge from Conduant –Some resistors on Amazon board also need changing; Conduant will update without charge Mark 5A/5B –Requires firmware version or higher  Appropriate bib file available on Haystack website Mark 5B+ –Requires SDK 8.1 or higher  Requires Flash_upgrade of Amazon card  Must be done manually; difficult to downgrade –At 2Gbps, have had best experience with Seagate disks SATA disks have been used in several major experiments; no significant problems in field or at Mark 4 correlator

PATA & SATA: $/GB vs. time courtesy Walter Brisken; data based on pricewatch.com monitoring

Update to Whitney 2001 $/GB plot

Mark5 OS Distribution Status –Next release will be Debian “Etch” distribution kernel currently under beta test Expect release ~end June 08 –Requires Conduant SDK8.1 Streamstor_source_v.8.1.deb (under test) –Mark5A/B/B+ software currently under test Target release is Mark5_source_1.0.deb (under test) –Mark5B/B+ interrupt handler under test Target release is mk5bio-dev-source_v.1.0.deb (under test)

iBob2/ROACH Under Development Joint development with NRAO, Haystack, UC Berkeley, South Africa UC Berkeley – concept development, hi-level FPGA design tools S. Africa – schematic design, initial checkout, PPC support NRAO – board layout, “VDBE” design 500 MHz BW IF input 4 DDCs/board; frequency agile; channel BW 64kHz to 256MHz Haystack – PFB “DBE2” design 2 x 1 GHz BW IF; adjacent channels; 8 Gbps aggregate output rate Haystack developing PFB application for ‘DBE2’ personality; NRAO developing DDC application for ‘VDBE’ personality; Prototypes expected 3Q 2008 Packaging: standard 1U PC server “pizza” box Cost <$10K First boards – May 2008; initial checkout ongoing S. Africa Expect first working VDBE, DBE2 prototypes ~end 2008

16 Gbps Burst-mode System Mark 5C 10GigE

Thank you!

Frequency source for mm/sub-mm VLBI Haystack working with Univ. of Western Australia to develop Cooled Sapphire Oscillator (CSO) technology for VLBI frequency reference CSO is as better than most H-masers to 100 seconds; ideal for mm/sub-mm observations Requires batched-cooled with liquid H2 Cost is of order $10,000 per unit (much cheaper than H-maser) Expect one CSO with GPS-conditioned, phase-locked 10MHz ref output available for VLBI in 2009

ROACH block diagram