1. LBTO software startup/shutdown and troubleshooting July 18, 2006 Chris Biddick 1 cjb

2. TCS servers lbtmu01 (TO station) lbtmu102 lbtmu103 lbtmu104 lbtmu105 lbtmu107 jet (special) 2 cjb

3. Start TCS networks netconfig start (on all servers, but not jet) networkserversubsystem manager rpcserverRPC manager gshmserverreflective memory manager netconfig start passive(TO station only) prevents subsystems from running here 3 cjb

4. Start MCSPU on jet User telescope rpcconfig start 10.144.0.20 [passive]./gotemp (used) Will start mcstemp (most recent control program) and the engineering interface program mcsDisplay../go (not used) Will start mcsworking (stable control program) and the engineering interface program 4 cjb

5. Start subsystems netconfig start [left | right] SUB Starts subsystem SUB on lightest load server netconfig start [left | right] SUB on address Starts subsystem SUB on server ‘address’ – name or IP address netconfig start [left | right] SUB on local Starts subsystem SUB attached to local terminal Alternate side notation: left SUBSUBL right SUBSUBR 5 cjb

6. Start subsystems When subsystems are started normally stdout is directed to /dev/null stderr is available (but not much use) When started ‘on local’ both stdout and stderr are available and attached to controlling terminal 6 cjb

7. Current subsystems LSSLogging control ECSEnclosure control CSQIIF receiver PMC leftPrimary mirror control PSF leftActive optics control MCSMount control PCSPointing control SDBStar database 7 cjb

8. Start GUIs GUIs are normal executables installed in a system path ECSGUI [READ_WRITE] ECSGUI [READ_WRITE] & For sided subsystems add ‘left’ or ‘right’ as argument: PSFGUI left 8 cjb

9. Current GUIs ECSGUI [READ_WRITE] PMCGUI {left | right} PSFGUI {left | right} MCSGUI (old) mcsgui (OpenGL) PCSGUI (old) pcsgui (OpenGL) presetgui (OpenGL) WXGUI 9 cjb

10. TO station start buttons ECSGUI TEL, ECS, ECSGUI READ_WRITE MATGUI CSQ, MATGUI MCSGUI MCS, mcsgui (OpenGL) PCSGUI PCS, pcsgui (OpenGL) PMCGUI TEL, LSS, left PMC, PMCGUI left PRESETGUI PCS, MCS, SDB, presetgui PSFGUI TEL, LSS, left PSF, PSFGUI left TELGUI TEL, TELGUI WXGUI ECS, WXGUI 10 cjb

11. Stop GUIs Use ‘File exit’ option or title bar ‘X’ 11 cjb

12. Stop subsystems netconfig stop [left | right] SUB Stops subsystem SUB – times out in seven seconds netconfig stop all Stops all subsystems netconfig kill [left | right] SUB Sends SIGKILL (9) instead of SIGINT (2) If GUI is left running when SUB stopped GUI should turn red 12 cjb

13. TO station stop buttons TCSstopall subsystems 13 cjb

14. Stop MCSPU on jet Command ‘stop’ in engineering interface will stop just the interface Command ‘stopall’ will stop both the engineering interface and mcstemp 14 cjb

15. Stop TCS networks netconfig stop (on each server) Stop all TCS network processes on local server You may stop TCS networks without stopping subsystems – network state is kept in files on disk 15 cjb

16. TCS state files LBT.conf Overall configuration parameters in /usr/commonsw/NetConfig rpc_IPAddr.conf RPC functions and aliases on local server in /var/state/LBT nameprocs.conf Subsystem running on local server in /var/state/LBT LSSEventClientList.dat List of event callbacks on local server in /var/state/LBT 16 cjb

17. Troubleshooting - netconfig netconfig ps Shows all TCS processes on local server netconfig –l Shows all known TCS subsystems on all servers netconfig –s Shows all servers with subsystem count and load 17 cjb

18. Troubleshooting - rpcconfig rpcconfig stop [address] Stops rpcserver on ‘address’ rpcconfig start address [cold] [passive] Starts rpcserver on ‘address’ ‘cold’ skips reading state file ‘passive’ disables rpc health check rpcconfig –l Shows number of registered functions and aliases on local server rpcconfig –f [address] Lists all functions on server address rpcconfig –a [address] Lists all aliases on server address 18 cjb

19. Troubleshooting - gshmconfig gshmconfig stop Stop gshmserver on local server gshmconfig start Start gshmserver on local server gshmconfig –b Shows reflective memory IP broadcast address on local server gshmconfig –l Lists segment key, size, and all servers currently on the reflective memory network Repeat and check that all servers stay at same ID 19 cjb

20. Troubleshooting - monitor monitor –o Shows computers as they join or drop off the shared memory network Shows restart requests 20 cjb

21. Troubleshooting - syslog /var/log/messages on local server ‘tcslog’ will show messages with LBT designation as they occur 21 cjb

22. Troubleshooting - DDViewerGUI DDViewerGUI shows hierarchical view of reflective memory on local server Available on every server and TO station Updates about every two seconds 22 cjb

23. Troubleshooting - Events No LSS - XML files on local server /tmp/SUB.log LSS - MySql database mysql –u lss –p –h mysql LBT_log password lss select date,description from Event where (date like ‘%Jul 18%’ and description like ‘%left PSF%’); 23 cjb

24. Mountain subsystem debugging Use your account on a server Get current CVS tree Modify subsystem code Build and install (may require removal of existing package) netconfig stop SUB netconfig start SUB on server 24 cjb

25. Mountain GUI debugging Use your account on a server or TO station Get current CVS tree Modify GUI code Build GUI (install not needed) Run GUI directly 25 cjb