1. Data centre support for the IGS-RT PP W. Söhne, H. Habrich, G. Weber Federal Agency for Cartography and Geodesy, Frankfurt am Main, Germany

2. Outline  Introduction  IGS-RT PP Data and product file centre File types under consideration  Technical aspects Data volume Up- and downloading Completeness Limitations  Policy aspects Upload policy  Conclusions & open questions

3. Introduction  Storing of highrate files nothing new for IGS At CDDIS back to 2001, doy121 Initiative from the LEO needs At IGN Others  Term “highrate” is a relative one Usually means 1 Hz sampling rate But higher sampling rates in use depending on the application  Currently ~110 stations at CDDIS  Clear statement from IGS concerning the needs of long-term archive still missing

4. IGS-RT PP (1)  One activity of the IGS-RT PP: Data and product file centre  File types under consideration Navigational data (broadcast ephemeris) Observational data -highrate RINEX files -hourly and daily RINEX files Derived files (products) -Orbits or orbit corrections -Clocks or clock corrections -Troposphere & ionosphere parameters

5. IGS-RT PP (2)  Why generation of highrate files? Currently, considerable number of users are “feeding” near real-time applications For future scientific analyses  What is “near real-time”? Typically hourly applications, with 20-30 minutes computation time  two times per hour computation possible  Why 15 minutes files? Reduction of file size compared to an hourly highrate file “Convention”

6. IGS-RT PP (3)  Is that interval small enough? Are there other intervals of interest, e.g. 5 minutes files? Advantages -Better distribution of uploading over time -Other intervals can easily be created/derived -File naming convention fits Disadvantages -Interval size not known in IGS -Number of files growing

7. Technical aspects: data volume  Data volume of highrate observational files Example BUTE115A45.08O: 1 Hz 15 minutes GPS+GLONASS RINEX file, # obs. types 8, 20 SV in view: 2.4 Mb Hatanaka+compress: 170 Kb  5.7 Gb per station and year for the compressed files  >> 500 Gb per year Selection of IGS stations for high-rate storing?  Data volume of product files Clocks: 1 Hz, 1 hour: 2.5 Mb, compressed 170 Kb  5.7 Gb per solution and year …

8. Technical aspects: up- and downloading  Data upload of observational files Example 250 stations, 24 files per day (hourly): 6000 files uploaded * 100 Kb  0.6 Gb per day Example 110 stations, 96 files per day (highrate): 10560 files uploaded * 170 Kb  1.7 Gb per day  Critical aspects are Is the data centre able to handle all incoming files within, e.g., 5 minutes after the full hour? If not  spreading of upload over a certain time span?! Is the data centre able to handle the parallel download requests in peak periods?

9. Technical aspects: completeness (1)  “Completeness” covers the aspects Number of observation types Number of epochs Number of SVs per epoch Ratio observed / predicted number of observations  Completeness can be affected by Outages of single stations data streams Outages of the broadcasting system Handling of unhealthy SVs Limitation of supported observation types

10. Technical aspects: limitations (1)  “Limitations” cover the following aspects Resolution of -Phase -Code Support of -GPS L2C, L5 -GLONASS -Galileo -SBAS

11. Technical aspects: completeness (2)  Different levels of comparison of original and accumulated files possible Character by character -Difficult due to roll-over phase values Completeness -Epochs missing? -SVs missing? Parallel analysis -Differences between the results

12. Technical aspects: completeness (3) Hourly file BUTE115A.08O (TEQC) Highrate file BUTE115A00.08O (BNC)

13. Technical aspects: limitations (2) ALME: file from TEQC vs. file from BNC (RTCM 2.3) after RNXSMT (BSW5.0)

14. Technical aspects: limitations (3) WTZR: file from TEQC vs. file from BNC (RTCM 3.0) after RNXSMT (BSW5.0)

15. Technical aspects: completeness (4) 579 8 0 1 1 0 58 56 6 2 99,9 -100,0% 99,5 - 99,6% 50,0 - 90,0% 100,0% 99,8 - 99,9% 90,0 - 99,0% < 50,0% 99,6 - 99,7% 99,0 - 99,5% 99,7 - 99,8% 168 5 0 0 1 0 21 35 21 99,9 -100,0% 99,5 - 99,6% 50,0 - 90,0% 100,0% 99,8 - 99,9% 90,0 - 99,0% < 50,0% 99,6 - 99,7% 99,0 - 99,5% 99,7 - 99,8% 01 0 0 1 0 4 14 2 2 99,9 -100,0% 99,5 - 99,6% 50,0 - 90,0% 100,0% 99,8 - 99,9% 90,0 - 99,0% < 50,0% 99,6 - 99,7% 99,0 - 99,5% 99,7 - 99,8% Completeness of 24 hourly high-rate RINEX files from RTIGS streams coming in via udpRelay Completeness of 233 hourly high-rate RINEX files from RTCMv2 streams coming in via NTRIP/TCP Completeness of 711 hourly high-rate RINEX files from RTCMv3 streams coming in via NTRIP/TCP

16. Technical aspects: completeness (5) Missing SVs G06, G21, G26, G30 Streaming interrupted Longer sections w/o GLONASS

17. Policy aspects (1) Station A Station B Station C Station N Data centre 1 Data centre 2 Data centre 3 User a User b User c User m Mirroring „Traditional“ situation transferring hourly files, daily files, ephemeris files  file creation at the station

18. Policy aspects (2) Possible situation using real-time data streams for RINEX file creation Station A Station B Station C Station N Broadcaster 1 User a User b Data centre 1 transferring files transferring streams or  file creation possible at the data centre, at the users site, at a third party

19. Conclusions & open questions (1)  Derivation of highrate RINEX files from real- time streams suitable tool  Current interval for highrate files 15 minutes – is that small enough?   Recommendation #1: Storing of 1 Hz 15 minutes files, at least for the long-term archive  Derivation of daily files from streams instead of ftp transfer, at least for new or proposed stations?  RT-IGS PP CfP found five candidates (BKG, CDDIS, GA, KASI, Univ. of Padova)

20. Conclusions & open questions (2)  Limitations acceptable? If yes, how to point the users to these limitations effectively?  Derivation of RINEX files from real-time streams possible for everyone – needs some regulation or clarification: Who is allowed to upload files derived from real-time data streams?   Recommendation #2: highrate file creation and upload to the GDC in one hand, ideally at the broadcaster’s side