The Mark 5B VLBI Data System Alan Whitney Will Aldrich John Ball Kevin Dudevoir Brian Fanous Dan Smythe MIT Haystack Observatory 24 March 2006 EVN TOG Dwingeloo

Current Mark 5A Status ~120 Mark 5 systems deployed; >95% of tape systems have been replaced >1000 Mark 5A ‘8-pack’ disk modules deployed (>2 PB of storage!); growing rapidly! Many stations are now entirely Mark 5A, except in some cases for data which goes to VLBA correlator 1 Gbps experiments are now routine for both geodesy and astronomy All 10 VLBA stations plus GBT are now converted to Mark 5A; VLBA correlator has 11 Mark 5A playback systems

Mark 5B Data System Features Full VSI-H compatibility; 64 MHz max clock rate Same chassis as Mark 5A; uses same disk modules; requires Mark 5B I/O card 1024 Mbps record/playback Eliminates need for external formatters, but requires sampler adapter for Mark 4/VLBA DASs to provide VSI-compatible output Station Unit capabilities for connection to Mark 4 correlators is designed into Mark 5B Extensive built-in phase-cal extraction and state counting on both data record and data playback Front-panel status display – 8 tri-color LEDs DIM and DOM capabilities are separate FPGA downloads FPGA is programmable via software Development support from Mark 5 development consortium – BKG, EVN, KVN, JPL, MPI, NASA, NRAO, USNO

VSI-H VSI-S VSI-E

Mark 5A/B System Comparisons Mk5AMk5B Data Interface Emulates Mk4/VLBA tape transport VSI-H (64MHz max clock rate) Max data rate 1024 Mbps Record modes 8, 16, 32, 64 “tracks” 1,2,4,8,16,32 bitstreams Disks Mk5 “8-pack”Same Chassis Mk5Same I/O card Mk5AMk5B SS card XF2 I/O-SS intf Modified FPDPFPDP

Mark 5B Functional Block Diagram

Mark 4 Station Unit emulation Mark 4 Station Unit

Mark 5B I/O Board FPDPVSI OutputVSI Cntl/Mntr VSI Input 256MB memory PCI Connector

Mark 5B Status Checkout of Mark 5B hardware is complete Mark 5B support software is functional but needs some additional capability and testing before release ~30 Mark 5B I/O boards have been built and tested – will be distributed to Mark 5 development consortia members in near future Design is being transferred to Conduant Corp for replication and sale – Mark 5B will be orderable from Conduant from 1 April 2006

Reasons to adopt Mark 5B Eliminate need for expensive external formatters; particularly important for new stations or stations without existing Mark 4 or VLBA formatters With a 14-BBC Mark4 or VLBA4 system, up to 1792 Mbps can be recorded with two parallel Mark 5B systems; current 14-BBC systems can only generate a maximum of 1024 Mbps of formatted data VLBA can expand max data rate from 512 Mbps to 1 Gbps with Mk5B Extensive phase-cal extraction and state counting capabilities for better diagnostics and better system calibration Replace unreliable Station Units on Mark 4 correlators; SU capability is built into Mark 5B

Mark 5A/B Compatibility Mark 5B can play only Mark 5B recordings (VSI format in/out) Upgraded Mark 5A (“Mark 5A+”) will be able to play: -All Mark 5A recordings -Mark 5B recordings made in selected modes; playback will be VLBA-track-format Mark 5A+ design is now ~3/4 complete – will be ready within a couple of months Bottom line: Existing Mark 4 correlators and VLBA correlator with only Mark 5A units will be able to process data from both Mark 5A and Mark 5B units during transition period to Mark 5B.

Mark 5B+ (2048Mbps) Conduant has introduced an upgraded StreamStor (dubbed “Amazon”) that supports up to ~3 Gbps on FPDP2 interface which allows clocking at 64 MHz Mark 5B I/O card has been designed to support input VSI-H clock rate of 64MHz, as well as FPDP2 compatibility, to support max recording rate of 2048 Mbps Testing with prototype Amazon board at Haystack is underway Recording at 2048 Mbps is across 2 disk modules (16 disks) simultaneously Playback is limited to 1024 Mbps Much software work and testing remains to be done No promises yet on date of general availability

Mark 5A/B/B+ System Comparisons Mk5AMk5BMk5B+ Data Interface Emulates Mk4/VLBA tape transport VSI-H (64MHz max clock rate) Max data rate 1024 Mbps 2048 Mbps Record modes 8, 16, 32, 64 “tracks” 1,2,4,8,16,32 bitstreams Same as Mk5B Disks Mk5 “8-pack”Same Chassis Mk5Same I/O card Mk5AMk5B SS card XF2 Amazon I/O-SS intf Modified FPDPFPDPFPDP2 (clocks on both edges

Mark 5 Upgrade Costs Target Existing Mk5AMk5BMk5B+ 0~$18K~$20K~$23K Mk5A-~$2.5K (Mk5B I/O) ~$12K (Mk5B I/O plus Amazon) Mk5B--~$9.5K (Amazon) Note: Does not include external cabling costs, typically a few hundred dollars

Disk-Media Status Hard disk price vs capacity/performance continues to drop -Now at or below ~$0.50/GB and continues to drop (Mark 4/VLBA tape is ~$2.00/GB) 250 GB disks now common – 8-pack of 250GB disks comparable to ~3.5 VLBA/Mark 4 tapes 400 GB disks are available – 8-pack of 400GB disks comparable to ~6 VLBA/Mark 4 tapes 700 GB disks expected by end 2005 – 8-pack of 700GB disks comparable to ~9 VLBA/Mark 4 tapes; ~24 1 Gbps unattended with 2 disk modules!

Tape vs. Disc Price Comparison

Disk-Media Reliability Based on statistics collected at Haystack, average disk drive failure rate is ~0.5% per year Failure rate of Hitachi 250GB has been higher than average Conduant has qualified drives from Maxtor, WD and Seagate 400GB – Seagate 250GB – Western Digital RAID Edition (high-reliability) 250GB – Maxtor Maxline III (high-reliability) Conduant is shipping these disks in assembled Mark 5 disk modules Disk reliability at high altitudes has been tested at summit of Mauna Kea – only Hitachi 250GB disks have performed reliably

Plans for Serial-ATA Support Desire to support SATA disk modules in Mark 5 – must be interchangeable with current PATA modules Unable to find SATA connectors with sufficient durability Plan to design PATA-to-SATA converters in into module backplane – no change in module connector Expect SATA modules to be available late 2005

Mark 5B Interface Boards VSI-4 sampler adapter board Will be placed inside Mark 4 formatter to upgrade to VSI Uses existing Mark 4 samplers and 1pps generator 2 VSI output connecters at 1024Mbps each (though data rate is restricted to 1792 Mbps by 14 BBC’s) Prototype boards ready for checkout VSI-C sampler adapter Interfaces VLBA samplers to VSI Designed at Metsahovi Correlator Interface Board (CIB) Interface between Mark 5B and Mark 4 correlator PCB currently in fabrication Upgraded Serial Link boards for Mark 4 correlator designed at MPI Prototypes have been successfully tested; replication to begin soon

VSI Interface Board for Mark 4 formatter

Mk4 Correlator Block Diagram

Mark 5B Correlator Interface Board (CIB)

Mark 5B Interfaced to Mark 4 Correlator with CIB

New Correlator Capabilities with Mark 5B Increased correlator throughput –32 MHz playback independent of record rate (if desired) –Inter-scan gap of ~5-10 seconds Increased reliability over SU –SU has 17 x 6U boards → 1 FPGA chip –Xilinx code rigorously simulated –Much simpler software than in SU Flexibility in FPGA design 16 tone x 16 channel phase cal ½ speed 2 Gb/s playback

Station Unit Problems Byte slips at 32 MHz playback rate causing amplitude loss Amplitude loss proportional to parity error rate Intermittent large pcal values Intermittent loss of fringes Intermittent crashes Slow (~30 s) sequencing for scan setup

Pcal Processing 16 tones (cf. 2 in SU) in each of 16 channels with 99.6% duty cycle 256 levels in phase-tone model Response to spurious signals > 40 dB

Fringes with Mark 5B GGAO to Westford test scan on a geodetic source –GGAO recorded in Mk5A format –Westford sampled and recorded on Mark 5B using Mk5 sampler module (Mk4 formatter + VSI4 board) Westford data file then converted by standalone “b2a” program to be Mk5A VLBA format (an easy conversion) Standard Mk4 correlation using 2 MK5A’s Within last week – direct correlation with Mark 5B with comparable results!

Future Plans Incorporation of 16 tone phase cal into fourfit –Statistical robustness to tone contamination –Phase corrections across band to increase sensitivity –Derive single-band delays per channel from multiband delay + pcal- derived channel delays should result in better coherence across channels. estimate only multi-band delay and delay rate Expansion of the Mk4 correlators –16 stations possible if sufficient # of Mk5B’s –Correlator control computer upgrade to modern PC + Linux

Summary Mark 5B is nearly ready for prime time Upgrading to Mark 5B is not currently necessary at many stations, but may be advantageous in certain situations, and will be mandatory with deployment of DBE Mark 5B has been successfully interfaced directly to Mark 4 correlator Mark 5B+ (2 Gbps) is under development Expect continuing upgrades to Mark 5 over the next few years