1. Station Operator support for a GCI VSAT link Efren Abaya Head GCI &Networks Operations Unit Operations Section, IDC Division Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization Provisional Technical Secretariat Vienna International Centre P.O. Box 1200 A-140 Vienna, AUSTRIA efren.abaya@ctbto.org

2. Provisional Technical SecretariatPage 2 Outline of Presentation Mission of the GCI network GCI Performance requirements Technologies & equipment used in the GCI GCI Network Monitoring System GCI Point of Contact – Roles and responsibilities Examples of GCI problems Abstract: The Global Communications Infrastructure (GCI) connects every IMS facility to the IDC with a target annual availability of 99.5%. Every GCI component is remotely monitored and controlled by the 24x7 Network Operations Centre in the USA as well as by PTS staff. There are times when local support is necessary to diagnose a failure condition or to restore GCI service in the most expedient way. This presentation will discuss the ways in which Station Operators can assist the PTS and the GCI Contractor in maintaining or restoring GCI service.

3. Provisional Technical SecretariatPage 3 CTBT, ARTICLE IV.A, paragraph 14a “… in cooperation with the State Parties the Technical Secretariat shall, for the purpose of this Treaty: (a)Make arrangements to receive and distribute data and reporting products relevant to the verification of this Treaty in accordance with its provisions, and to maintain a global communications infrastructure appropriate to this task;...” GCI = Global Communications Infrastructure Mission of the GCI Network

4. Provisional Technical SecretariatPage 4 Relevant GCI performance requirements - Virtual Circuit Availability VSAT – must be up 99.5% of the time over any consecutive 365 days  total outage time less than 44 hours per year Terrestrial – 99.95% over any consecutive 365 days  total outage time less than 4.4 hours per year Backup Power for AC outage 2 to12 hours of UPS power backup GCI Performance Requirements

5. Provisional Technical SecretariatPage 5 GCI VSAT link at an IMS station or NDC Other station or NDC equipment

6. Provisional Technical SecretariatPage 6 Typical GCI equipment at IMS station/NDC

7. Provisional Technical SecretariatPage 7 C-band 2.4 meter VSAT antenna (Prodelin) ODU (outdoor unit) BUC (Block Upconverter) LNB (Low-noise Block Downconverter) Feedhorn OMT (Orthomode transducer)

8. Page 8 Ku-band 1.2 meter VSAT antenna (Prodelin) ODU (outdoor unit) BUC LNB Feedhorn OMT Notice the yellow grounding wire at the bottom of the pole mount Provisional Technical Secretariat

9. Status, Traffic, Round Trip time, Satellite signal quality, UPS power, etc. Traverse Network Management System (NMS) remotely monitors every GCI link Page 9Provisional Technical Secretariat

10. Page 10 Cooperative arrangements for GCI Operations &Maintenance CTBTO GCI &Network Operations Unit IMS IT & GCI Support Unit & PTS Operations Centre GCI Contractor Ultisat (NOC) GCI Point of Contact (POC) at stations & NDCs IDC GCI&NO Unit deals with day-to-day performance issues and reporting. IMS IT&GCI Support Unit deals with installations and infrastructure issues.

11. Provisional Technical SecretariatPage 11 Responsibilities of the GCI Point of Contact (POC) 1.FORMALITIES - the person(s) who communicates at the highest level regarding the GCI link. For example, this POC nominates the other two types of POC and informs the CTBTO of any changes in POC assignments. Communications at this level will be rare. 2.MAINTENANCE - the POC(s) whom we contact for day-to-day issues with the GCI link. For example, this POC might be requested to restart some equipment or check the conditions at the site. This would also be the person whom the GCI Contractor contacts if they need to send their maintenance engineer to the site. Have more than one person in this position because it is unavoidable that staff have to leave the office from time to time. Normally, this is the same person(s) as the station local operator. 3.SHIPPING ADDRESS - the person that the PTS or the GCI Contractor contacts if some equipment or spare parts need to be shipped to your site. Delivery companies usually require a street address and a telephone number (and if possible a person’s name) to deliver door-to-door.

12. Examples of tasks on Request Report LED and visual status indicators of iDirect satellite modem, Cisco router, UPS, antenna de-icer if applicable Verify that cables and connectors are in place; inspect for damage Inspect the Line of Sight (LOS) of the VSAT dish antenna Clean the dish antenna and/or feedhorn cover if necessary Power cycle (reset) equipment Carry out tests on GCI equipment, such as UPS Swap out indoor unit, router, BUC/LNB (in some cases) with cold spare unit Remote Site Maintenance – Examples of tasks on request by the NOC (1/2) Page 12Provisional Technical Secretariat

13. Verify power connection; check presence of AC/DC power to UPS, UPS battery status Report (on request) unusual conditions such as ice, snow, heavy rain Provide access to site for PTS/contractor maintenance personnel Logistical assistance for travel by PTS or GCI contractor personnel to site Logistical assistance for shipment of replacement parts to site, or returning spares to GCI Contractor Assist in renewing VSAT operating license (when requested) Normally, Ultisat takes care of this silently in the background Other tasks of a similar nature to the above as requested by the PTS or the GCI Contractor Remote Site MaintenanceMaintenance – Examples of tasks on request by the NOC (2/2) Page 13Provisional Technical Secretariat

14. Storage of GCI spares Provide secure storage for spare GCI equipment (electronics) Know where spares are kept! Examples of Periodic Tasks Check the line of sight (LOS) from antenna to satellite; trim trees if applicable Verify no physical or mechanical damage to GCI equipment and cables Report if the antenna or platform or mount is unstable or is damaged Clean the antenna dish, if needed (for example, to remove snow) Keep GCI documentation available for use of station personnel Turn antenna de-icer on or off every season Remote Site Maintenance -- Examples of periodic or routine tasks Page 14Provisional Technical Secretariat

15. Page 15 Causes of GCI communication problems Breakdown of equipment – router, modem, outdoor unit component, connecting cables and connectors Loss of AC or DC mains power supply Frequent power outages shorten the battery life Maintenance work at a station or NDC Line of sight (LOS) blockage Risks to the equipment: Cutting grass, digging around the antenna, building new structures near the antenna Vandalism or theft of equipment Animals – birds, rats, dogs, horses

16. Water droplets weaken the radio signal from the satellite. Inform the NOC if this happens. Send photgraphs. Water or moisture in the feedhorn Page 16 GOOD BAD Provisional Technical Secretariat

17. Feedhorn cover taped to stop entry of water. Recommended first aid: Cover the entire feedhorn with a plastic bag. Bird damage to feedhorn cover and IFL cable Page 17Provisional Technical Secretariat Damaged IFL cable sheath should be taped to protect against water entry.

18. LOS = Line of sight Circle marks where the satellite is in the sky Before and after clearing Line of Sight (LOS) Page 18Provisional Technical Secretariat AFTER BEFORE

19. Photo or verbal description of the condition of status lights helps the GCI Contractor in troubleshooting. Reporting status lights to the NOC Page 19Provisional Technical Secretariat iDirect modem iDirect router Cisco router

20. Water, snow and ice weaken the radio signal from the satellite. Remove the snow, if possible. Inform the NOC if this happens. Snow on the antenna Page 20Provisional Technical Secretariat

21. Inform the NOC if the IMS facility cannot be reached to service the GCI equipment. Access problems Page 21Provisional Technical Secretariat

22. Corrosion of bolts and metal supports at the VSAT antenna – Inform the NOC Page 22Provisional Technical Secretariat BAD GOOD Denso tape on metal support rod Minor surface rust is normal

23. Rat droppings in failed GCI equipment Page 23Provisional Technical Secretariat

24. Page 24 NOC email to POC – Unknown reason From: GCI NOC [mailto:noc_gci@ultisat.com] Sent: 20 June 2012 16:58 To: noc_gci; + POC addresses Cc: 'gcintt@ctbto.org'; EHD; Raymond Nyamande Subject: GL01 Link Outage - iHeat Ticket 435865 Dear Sir/Madam, We have noted an outage at your site. Site Code: GL01 Outage Time (UTC): 16Jun12 23:51 UTC Ticket Number: 435865 If any of your staff has access to the GCI equipment at the site that can inform us as to why the site is down, we would greatly appreciate it. We appreciate the time that you spend on GCI matters and thank you in advance for your co-operation. Kindly reply to all the email addresses above. Please quote the number displayed in the subject line in all communications about this particular outage. Please be advised we will call you at the phone numbers below during your normal working hours if the site does not clear before then to discuss how we may assist you with this outage. POC Name POC Office Number POC Mobile Number Mr Gibson Mponela +431 26030 0 If any of the above numbers are incorrect, please advise the NOC by replying to all above. Regards,

25. Provisional Technical SecretariatPage 25 NOC email to POC – Weather-related outage From: GCI NOC [mailto:noc_gci@ultisat.com] Sent: 20 June 2012 16:58 To: noc_gci; + POC addresses Cc: 'gcintt@ctbto.org'; EHD; Raymond Nyamande Subject: GL01 Link Outage - iHeat Ticket 435865 Dear Sir/Madam, We have noted an outage at your site. Site Code: GL01 Outage Time (UTC): 16Jun12 23:51 UTC Ticket Number: 435865 If any of your staff has access to the GCI equipment at the site that can verify that the site is down due to weather conditions in the area, we would greatly appreciate it. We appreciate the time that you spend on GCI matters and thank you in advance for your co-operation. Kindly reply to all the email addresses above. Please quote the number displayed in the subject line in all communications about this particular outage. Please be advised we will call you at the phone numbers below during your normal working hours if the site does not clear before then to discuss how we may assist you with this outage. POC Name POC Office Number POC Mobile Number Mr Gibson Mponela +431 26030 0 If any of the above numbers are incorrect, please advise the NOC by replying to all above. Regards,

26. Provisional Technical SecretariatPage 26 NOC email to POC – Power outage From: GCI NOC [mailto:noc_gci@ultisat.com] Sent: 20 June 2012 16:58 To: noc_gci; + POC addresses Cc: 'gcintt@ctbto.org'; EHD; Raymond Nyamande Subject: GL01 Link Outage - iHeat Ticket 435865 Dear Sir/Madam, We have noted an outage at your site. Site Code: GL01 Outage Time (UTC): 16Jun12 23:51 UTC Ticket Number: 435865 If any of your staff has access to the GCI equipment at the site that can verify that the site is down due to power, we would greatly appreciate it. We appreciate the time that you spend on GCI matters and thank you in advance for your co-operation. Kindly reply to all the email addresses above. Please quote the number displayed in the subject line in all communications about this particular outage. Please be advised we will call you at the phone numbers below during your normal working hours if the site does not clear before then to discuss how we may assist you with this outage. POC Name POC Office Number POC Mobile Number Mr Gibson Mponela +431 26030 0 If any of the above numbers are incorrect, please advise the NOC by replying to all above. Regards,

27. Provisional Technical SecretariatPage 27 Reporting or responding to a suspected GCI problem Open an IRS problem report (for a new issue) http://irs.gci.ctbto.org needs a GCI connection http://irs.gci.ctbto.org IRS client sends an email problem report over Internet If above two don’t work, send an email problem report to support@ctbto.org and noc_gci@ultisat.com support@ctbto.orgnoc_gci@ultisat.com Include details: Describe the symptoms of the problem as you see it What equipment is affected (specify name or IP address) When it happened, or when it started & stopped (date & time) Is it still happening? The cause of the problem (if you know it) If relevant – Photographs – The current version of IRS does not support attachments therefore pictures should be sent to gcintt@ctbto.org with reference to the Problem Report ID (if there is an open ticket).gcintt@ctbto.org

28. Provisional Technical SecretariatPage 28 Thank you September.2015