1. 2012 DiFX user’s meeting, Sydney, Australia USNO Software Correlator: Status Report Outline USNO/WACO Background USNO Prototype Software Correlator (UPSC) Use of DiFX Recent Correlator Developments/Additions Dave Boboltz, John Spitzak, Nicole Geiger, Dave Hall, Alan Fey, Chris Dieck

2. 2012 DiFX user’s meeting, Sydney, Australia USNO’s Role in the Geodetic Community USNO is the International Earth Rotation and Reference Systems Service (IERS) Rapid Service/Prediction Center for Earth Orientation. USNO has several functions within the International VLBI Service for Astrometry and Geodesy (IVS). –Correlator Operations Center. –VLBI Data Analysis Center. USNO provides several levels of data products. –Daily EOP (rapids). –Weekly EOP updates (Bulletin A). –Long-term (periodic EOP, TRF, CRF solutions).

3. 2012 DiFX user’s meeting, Sydney, Australia Geodetic VLBI and the IVS Geodetic VLBI is coordinated through the IVS. http://ivscc.gsfc.nasa.gov/

4. 2012 DiFX user’s meeting, Sydney, Australia WAshington COrrelator (WACO) Current MarkIV “hardware” correlator was designed and constructed by MIT Haystack Observatory and is operated in cooperation with NASA. 1 of 4 IVS correlators –WACO, Haystack, Bonn, Japan. Only 2 MarkIV correlators still in operation. Experiencing end-of-life issues. Move to software-based correlator.

5. 2012 DiFX user’s meeting, Sydney, Australia WACO Workload Daily “Intensive” experiments –Used to measure variable rotation rate of the earth (UT1-UTC). –7 days/week: single-baseline, 1-hour experiment. –Occasional 3-station intensives. VLBA (MK-PT) intensives. –1-hr / day, 7 days / week Weekly R4 experiments –Used to measure EOP (polar motion, nutation, precession). –1 per week: 5-8 stations, 24-hr experiment. Other experiments –1 per month: 2-4 stations, 24-hr CRF experiments. –4 to 6 per year: 24-hr “R1” experiments backing up Bonn. –Occasionally: other 24-hr experiments (APSG, CONT, etc).

6. 2012 DiFX user’s meeting, Sydney, Australia USNO Prototype Software Correlator Heterogeneous cluster. –5 nodes (workstations) –32 cores 2 Mark5 data playback units. 1 Gb/s ethernet switch. DiFX package installed. Proof of concept. –Correlates geodetic VLBI data. –Correlates MK-PT Intensives. Procurement of full software correlator cluster.

7. 2012 DiFX user’s meeting, Sydney, Australia DiFX Processing of Geodetic Intensives Single baseline KK-WZ Currently correlating Friday intensive in parallel with h/w correlator E-transferred data written to Mark5 modules Correlated on UPSC difx2mark4 Fringe-fitting in HOPS/Fourfit. Geodetic databases created.

8. 2012 DiFX user’s meeting, Sydney, Australia USNO-NSF-AUI/NRAO MoA USNO contracting with NRAO for: –Access to MK - PT baseline. 1 hr intensive. 7 days/week. –Continued VLBA participation in the RDV experiments. Six 24-hr experiments per year. Vital to the CRF and TRF. Requires high-speed connections.

9. 2012 DiFX user’s meeting, Sydney, Australia Status: MK-PT Fiber Links Pie Town link (1 Gbps). –Available March 2011. –Numerous transfers of VLBI data. –100 - 300 Mbps to USNO. –Contract with NRAO. FY13 Contract in process. Mauna Kea link (1 Gbps). –Installed and available July 2011. –Numerous transfers of VLBI data. –100 - 300 Mbps to USNO. –Contract with Univ. of Hawaii. FY13 contract signed.

10. 2012 DiFX user’s meeting, Sydney, Australia VLBA MK-PT Intensive Experiments IVS Intensive4 (INT4) series –IVS Nyyddd Dual S/X band –4 S-band, 12 X-band –32 MHz/channel ~40 min. / experiment, ~30 scans 16 second scan lengths –Limits data to be transferred –Source lists from USNO, ICRF def. Data fringed and analyzed. –Oct. 2011 – May 2012

11. 2012 DiFX user’s meeting, Sydney, Australia Leap Seconds UT1-UTC Results

12. 2012 DiFX user’s meeting, Sydney, Australia UT1–UTC Results: Comparison IERS C04 Oct. 2011 – May 2012. IVS 24-hr sessions (red). IVS Intensives (green). –wrms = 16.1  sec. MkPt Intensives (blue). –wrms = 54.5  sec. –Un-modeled motion likely in Pie Town antenna.

13. 2012 DiFX user’s meeting, Sydney, Australia New USNO Software Correlator USNO contracted with NRAO to build correlator. –Phases I & II –Leverage NRAO DiFX/Cluster expertise Phase I (Sept. 2012). –Replacement for current 15 station hardware correlator. –30 compute nodes. –16 cores/node = 480 cores –Delivered to USNO Friday! Phase II (2013 - 2014). –Correlator expansion. –30 station VLBI2010 compatible

14. 2012 DiFX user’s meeting, Sydney, Australia Phase I Correlator 2 infrastructure nodes 1 head node 30 compute nodes –Diskless –Mount operating system and DiFX from infrastructure nodes 10 Gbps Ethernet network –Arista 10GbE switch –64 SFP+ ports –40 Gbps bonded connection to SAN

15. 2012 DiFX user’s meeting, Sydney, Australia Phase I at NRAO, Socorro 7 node Test Cluster will remain at NRAO 33 node Production Cluster

16. 2012 DiFX user’s meeting, Sydney, Australia Phase I Delivered to USNO Mark IV correlator Mark III correlator Seismic rated rack Cluster hardware

17. 2012 DiFX user’s meeting, Sydney, Australia Future Plans Get the new correlator up and running –James Robnett (NRAO) visit to USNO 24-28 Sept. –Unpack, rack, power-up, network connections. –Connect to SAN. –Test correlations off SAN. Connect Mark5 units via 10GbE –Emulex OCe10102 10 Gbps iSCSI network adapters installed. –6 Mark5 units available now. Parallel operations. –Run both correlators in parallel for ~6 months. –Compare results. –Phase out Mark IV correlator Go operational with MK-PT Intensives