1. NextGenerationCorrelatorWorkshop@Groningen NextGenerationCorrelatorWorkshop@Groningen 2006.6.28 2006.6.28 Kondo, T. *1*2 and T. Hobiger *2 Kondo, T. *1*2 and T. Hobiger *2 *1 Kashima Space Research Center, *1 Kashima Space Research Center, National Institute of Information and National Institute of Information and Communications Technology, Japan Communications Technology, Japan *2 Vienna University of Technology *2 Vienna University of Technology Kondo, T. *1*2 and T. Hobiger *2 Kondo, T. *1*2 and T. Hobiger *2 *1 Kashima Space Research Center, *1 Kashima Space Research Center, National Institute of Information and National Institute of Information and Communications Technology, Japan Communications Technology, Japan *2 Vienna University of Technology *2 Vienna University of Technology An XF-type Software Correlator Developed by NICT

2. Contents Software correlators at KashimaSoftware correlators at Kashima History of the processing speed of XF- type software correlatorsHistory of the processing speed of XF- type software correlators Details of an XF-type software correlator for geodetic useDetails of an XF-type software correlator for geodetic use Distributed processingDistributed processing ConclusionsConclusions Software correlators at KashimaSoftware correlators at Kashima History of the processing speed of XF- type software correlatorsHistory of the processing speed of XF- type software correlators Details of an XF-type software correlator for geodetic useDetails of an XF-type software correlator for geodetic use Distributed processingDistributed processing ConclusionsConclusions

3. Software Correlators at NICT *This software correlator is very fast, but not for geodetic use right now K5/VSSP K5/VSI for K5/VSSP (geodetic use) for K5/VSI (gigabit VLBI system) FX-Type Correlator (developed by Kimura-san)* XF-Type Correlator (for 1bit AD data) (K5-cor) FX-Type Correlator (for multi-bit AD data) (K5-fx_cor)

4. History of the processing speed of XF-type software correlators

5. History of Processing Speed (XF-type) (converted to 32-lag complex correlation) Throughput (bps) Year 19601970198019902000 2010 1M 10M 100M 1G 100K 10K 1K 100 Mark-I (IBM360/50) K-1 K-3 (mini-computer)

6. CONTROL COMPUTER TELEPHONE LINES COMPUTER MT HOST COMPUTER HP-1000 45F RECEIVER K-3 FORMATTER K-3 TAPE K-3 DECODER (1Mbit BUFFER MEMORY) MODEM RECEIVER K-3 FORMATTER K-3 TAPE K-3 DECODER (1Mbit BUFFER MEMORY) CONTROL COMPUTER CONTROL COMPUTER COMPUTER MT MODEM KASHIMAMIYAZAKI Historical e-VLBI Experiment using Software Correlator in JAPAN (1986) 1200bps It took about 10 hours to get fringes! KASHIMA MIYAZAKI 1000km

7. History of Processing Speed (XF-type) (converted to 32-lag complex correlation) Throughput (bps) Year Mark-I K-1 Moore's Law K-3 (IBM360/50) 19601970198019902000 2010 1M 10M 100M 1G 100K 10K 1K 100 (mini-computer)K-5 (PC)

8. Details of an XF-type software correlator for geodetic use

9. Requirements for a Software Correlator for Geodetic Use Compatible with conventional hardware correlators, such as K3, KSP correlators –Consistent definitions in delay, clock parameters, etc. –PCAL signal detection –Checking bit slip or make Both K5 and Mark-5 data processing capability

10. Architecture of K5-cor Correlator Imagcorr X(t) Y(t) fringe stopping cos corr. N lag sin corr. N lag Realcorr X-Pcal detection Y-Pcal detection X Pcal phase Y Pcal phase header check header check delay tracking

11. How to get 8-lag cross-correlation 1Byte （ 8bits) data refer look-up table as a function of 1byte-integer(X) and 2byte-integer(Y) 10011010 MSBLSB = 128+32+8+1= 169 00011011 MSB LSB = 32768+8192+4096+1024+128+64+16+4+1= 46293 X Y correlation counts for 0 lag = cortable0(169,46293) =6 10111010 16bits data × ○○○○○ × ○

12. How to get 8-lag cross-correlation 1Byte （ 8bits) data refer look-up table as a function of 1byte-integer(X) and 2byte-integer(Y) 10011010 MSBLSB = 128+32+8+1= 169 00011011 MSB LSB = 32768+8192+4096+1024+128+64+16+4+1= 46293 X Y correlation counts for 0 lag = cortable0(169,46293) =6 1 lag = cortable1(169,46293) =2 10111010 16bits data × ○ × ○ ××× ×

13. How to get 8-lag cross-correlation 1Byte （ 8bits) data refer look-up table as a function of 1byte-integer(X) and 2byte-integer(Y) 10011010 MSBLSB = 128+32+8+1= 169 00011011 MSB LSB = 32768+8192+4096+1024+128+64+16+4+1= 46293 X Y correlation counts for 0 lag = cortable0(169,46293) =6 1 lag = cortable1(169,46293) =2 ……………………. 7 lag = cortable7(169,46293)=0 10111010 16bits data ××××××××

14. Actual look-up table is cortable(169,46293)= table size (8lags) =2 8 x2 16 x4 bytes =67Mbytes table size (16lags) =2 8 x2 32 x8 bytes =8.8Gbytes too large 00010101................00...00. lag 0lag 1lag 7 32-bit integer lag 2lag 3lag 4lag 5lag 6

15. Mark5 to K5 converter (m5tok5) Analyze bit length, bit position vs track# Analyze bit length, bit position vs track# CIFCIF VEX File Mark5 File ConversionInformationFile CIFCIF Get AD bits, fanout, track#, CH# informationGet AD bits, fanout, track#, CH# information Mark5  K5 conversion conversion K5 File information file creation mode information file creation mode conversion mode

16. ・・・・・・・ LSB bit#0 MSB bit#31 bit #n Mark5 Data File Format Trk #k

17. ｔ ・・・ Mark IV ・・・ header 160bits data 20000bits 20160bits/frame VLBA ・・・ sampled data header 160bits data 19840bits 20000bits/frame

18. Mark IV Header block format block# word# byte# bit# 1 2 3 4 5 7 6 8 P 1234 1 1234 2 1234 3 1234 5 1234 4 12 1 ・・・ 5 ・・・・ Auxiliary Data (16 hexa) Sync word (all “1”) Y 100DD 10H10D HM 10M S 10S 0.01S 0.1S0.001S CRC -12 code Sampled data (20byte) ・・・・ odd parityeven (“0”)odd parityodd Auxiliary fieldSync wordTime field Header block Data block FRAME BORDER

19. VLBA Header block format block# word# byte# bit# 1 2 3 4 5 7 6 8 P 1234 1 1234 2 1234 3 1234 5 1234 4 12 1 ・・・ 5 ・・・・ Auxiliary Data (16 hexa) Sync word (all “1”) 10MJD10000S 1000SMJD 100SS 10S 0.01S 0.1S 0.0001S 0.001S CRC -16 code Sampled data (20byte) ・・・・ odd parityeven (“0”)odd parityodd Auxiliary fieldSync wordTime field Header block Data block 100MJD FRAME BORDER

20. K5/VSSP Data File Format (in case of 4ch data)

21. K5/VSSP Data File Format sampled dataheader data ｔ (in case of 1ch data)

22. K5 － Mark5 Fringes Oct. 15, 2002 Kashima - Westford X bandS band 4C39.25

23. Distributed processing

24. VLBI@home Distributed Correlation Processing Server huge VLBI data developed by Takeuchi-san GSI has developed similar distributed system

25. Improvement Factor Total Processing Speed using N PCs Total Processing Speed using one PC = Assumptions –Data are transferred from a server –Data transfer does not affect the speed of correlation processing –Correlation processing is possible with a data transfer for a different scan –Speed of data transfer reduces to 1/N in case of N parallel data transfers

26. Improvement Factor k (=network speed / processing speed) 10 scans N=20 N=10 N=2 100Mbps 1Gbps Total Processing Speed using N PCs Total Processing Speed using one PC =

27. Total Processing Speed in case of the use of N PCs Network Speed >> Processing Speed Network Speed ~ Processing Speed N 1 3 2 4 N times faster same as one PC

28. Current Status of XF Software Correlator Correlation processing speed of K5-corCorrelation processing speed of K5-cor –17Mbps for 32-lag complex correlation –34Mbps for 16-lag* complex correlation Network speed for distributed processingNetwork speed for distributed processing –100Mbps ~ 1Gbps (at office), 10Gbps (available) Total processing speed of distributed processing can beTotal processing speed of distributed processing can be –~170Mbps (32-lag, use of 10 PCs) –~340Mbps (32-lag, use of 20 PCs) –~1.7Gbps (32-lag, use of 100 PCs) –~3.4Gbps (32-lag, use of 200 PCs) Correlation processing speed of K5-corCorrelation processing speed of K5-cor –17Mbps for 32-lag complex correlation –34Mbps for 16-lag* complex correlation Network speed for distributed processingNetwork speed for distributed processing –100Mbps ~ 1Gbps (at office), 10Gbps (available) Total processing speed of distributed processing can beTotal processing speed of distributed processing can be –~170Mbps (32-lag, use of 10 PCs) –~340Mbps (32-lag, use of 20 PCs) –~1.7Gbps (32-lag, use of 100 PCs) –~3.4Gbps (32-lag, use of 200 PCs) * 16-lag is good enough for geodetic VLBI more realistic

29. 198020002010199020202030 1M 1G 1T 1P LAN WAN YEAR SPEED (bps) Road Map of Network Speed and K5-cor Processing Speed 10G 100G WAN & LAN from http://www8.cao.go.jp/cstp/project/super/haihu02/siryo3-sanko.pdf K5-cor ?

30. Expectation at 2010 Correlation processing speed of K5-corCorrelation processing speed of K5-cor –~100Mbps for 32-lag complex correlation Network speed for distributed processingNetwork speed for distributed processing –~100Gbps Total processing speed of distributed processingTotal processing speed of distributed processing –~ 1Gbps (32-lag, use of 10 PCs) –~ 2Gbps (32-lag, use of 20 PCs) –~ 10Gbps (32-lag, use of 100 PCs) –~ 20Gbps (32-lag, use of 200 PCs) Correlation processing speed of K5-corCorrelation processing speed of K5-cor –~100Mbps for 32-lag complex correlation Network speed for distributed processingNetwork speed for distributed processing –~100Gbps Total processing speed of distributed processingTotal processing speed of distributed processing –~ 1Gbps (32-lag, use of 10 PCs) –~ 2Gbps (32-lag, use of 20 PCs) –~ 10Gbps (32-lag, use of 100 PCs) –~ 20Gbps (32-lag, use of 200 PCs) more realistic

31. Conclusions Processing speed of software correlator (K5-cor : XF type) for geodetic use is 17Mbps for 32-lag complex correlationProcessing speed of software correlator (K5-cor : XF type) for geodetic use is 17Mbps for 32-lag complex correlation 170Mbps~340Mbps for distributed processing with 10~20 PCs will be available soon170Mbps~340Mbps for distributed processing with 10~20 PCs will be available soon About 10~20Gbps for distributed processing with 100~200 PCs will be expected by 2010, if we don’t care about “costs” and “save-energy”.About 10~20Gbps for distributed processing with 100~200 PCs will be expected by 2010, if we don’t care about “costs” and “save-energy”. Processing speed of software correlator (K5-cor : XF type) for geodetic use is 17Mbps for 32-lag complex correlationProcessing speed of software correlator (K5-cor : XF type) for geodetic use is 17Mbps for 32-lag complex correlation 170Mbps~340Mbps for distributed processing with 10~20 PCs will be available soon170Mbps~340Mbps for distributed processing with 10~20 PCs will be available soon About 10~20Gbps for distributed processing with 100~200 PCs will be expected by 2010, if we don’t care about “costs” and “save-energy”.About 10~20Gbps for distributed processing with 100~200 PCs will be expected by 2010, if we don’t care about “costs” and “save-energy”.