1. Workshop in ALMA Logs Prepared by Juan Pablo Gil – Arturo Hoffstadt
ADC Software Group
ALMA Observatory
March, 2015

2. ALMA Hardware Layout 66 Antenna ~45 device per antenna AOSTiming
Photonic References
64 ant-correlator
16-antenna correlator

3. ALMA Software Layout ALMA Software is built upon ACS
Different Subsystems
CONTROL
Antennas
Time reference
SCHEDULING
DATACAPTURER
BLCORR (64 ant)
ACACORR (16 ant)
TELCAL
QUICKLOOK
EXECUTIVE
ARCHIVE

4. ACS Deployment dv01-abm Component Container Host
Actual code that controls one physical device, or does one specific task
Eg. IFproc, DTX, ARCHIVE_CORR_BULKRECEIVER_NT, CONTROL_MASTER
Container
Logic grouping of components.
Has PATH/containerName
Eg. CONTROL/DV01/cppContainer has IFProc, DTX
Host
Computer which runs containers.
Eg. dv01-abm runs CONTROL/DV01/cppContainer
xx-abm, gasXX, cob-cpn-xx, lo-lmc-x, etc
dv01-abm
CONTROL/DV01/cppContainer
IFProc
ADC
Mount
WCA
Other containers…

5. ALMA Logging
Human-readable description of one action at one specific time
~30 GB of every day log
Container Logs:
Coarse grained (Info)
Local to machine
Stored in plain text files
XML Logs:
Fine grained (Debug)
System wide
Stored in XML files
Introducing high performance distributed logging service for ACS - Avarias et all, 2010

6. XML Log Example A typical XML log entry
<Debug
TimeStamp=" T17:14:50.658" File="WVRImpl.cpp” Line="665” Routine="void WVRImpl::publishWVRData(const Control::WVRData#, ACS::Time)” Host="pm01-abm" Process="CONTROL/PM01/cppContainer" Thread="CONTROL/PM01/WVRMonitorThread" Context="” SourceObject="CONTROL/PM01/WVR" Audience="Developer” >
<![CDATA[Sucessfully published the following WVR event. Antenna: PM01 Mid-point of data: 17:14: Temperatures: , , , deg. K. Sampled every seconds. Event is valid. Data monitored at: 17:14:50.496]]>
</Debug>
Log Levels: TRACE, DEBUG, INFO, NOTICE, WARNING, ERROR, EMERGENCY, …
2.5 millions of XML entries in 4 hours of observatory operation.
Valuable information hidden in Debug and below.
Main tool: jlog

7. Container Log Example Subset of XML log.
Bird of Eye view of operation (By default, only stores up to INFO level)
Generated at container level as text log line.
T11:52: [CONTROL/DV10/FrontEnd/ACD - void ACDImpl::setCalibrationDeviceBandInternal(CalibrationDeviceMod::CalibrationDevice, Receiver
BandMod::ReceiverBand)] Setting Hot load in front of band 4
Vector Size: 1
Starttime: 1e4ce2e5e639700
Band: 3
Freq: e+10
Caldevice 2
T11:52: [CONTROL/DV10/FrontEnd - void FrontEndImpl::lockFrontEndInternal(Control::FrontEnd::SubscanInformation, bool, bool)] Receiver
band was already locked at e+11
ACD Movement: Time: e602f8 To: 3 Set As: 2
ACDCSV,e602f8,3,2
T11:52: [CONTROL/DV10/cppContainer-GL - void PositionStreamConsumer::processData(Control::MountStatusData)] At 11:52: got the l
ast needed data with a timestamp of 11:52: The delay is seconds.
T11:52: [CONTROL/DV10/MountController - virtual void Control::MountControllerImpl::abortDataCollection()] Cleaning up past data colle
ction that started at 11:51: and was due to stop at 11:52: Time now is 11:52:16.253

8. What is not an ALMA Log
It is not observational data. It is behavior description.
Linux messages: dmesg, /var/log/messages
Mount circular buffers
BL CORR internal coding
Engineering scripts
HVAC shutdowns
Utility Modules

9. Where are the Logs? Container logs /alma/logs/{hostname} XML Logs
/mnt/gas02/data1/AcsLogs-8.1/
Don´t be afraid of the name!
log T19:42:07.557_ T20:03: xml
Other places to search
Computing Logs:
Kibana (beta):

10. Useful locations at AOS
CONTROL
/alma/logs/gas01/CONTROL/ACC/javaContainer
/alma/logs/gas01/CONTROL/ACC/pythonContainer
Antennae
/alma/logs/ANT-abm/CONTROL/ANT/cppContainer
Photonic References
/alma/logs/lo-lmc-1/CONTROL/LMC1/cppContainer
/alma/logs/lo-lmc-2/CONTROL/LMC2/cppContainer
BL Corr
/alma/logs/cob-cdpm-b/CORR/CDPMaster/cppContainer
/alma/logs/cob-cc/CORR/CCC/cppContainer
ACA Corr
/alma/logs/coj-cc-1/ACACORR/OBSERVATION_CONTROL/cppContainer
/alma/logs/coj-cc-1/ACACORR/CMD_TRANSFER/cppContainer
/alma/logs/coj-cpm-1/ACACORR/CDPM/cppContainer

11. Thank you!
The Atacama Large Millimeter/submillimeter Array (ALMA), an international astronomy facility, is a partner- ship of Europe, North America and East Asia in cooperation with the Republic of Chile. ALMA is funded in Europe by the European Organization for Astronomical Research in the Southern Hemisphere (ESO), in North America by the U.S. National Science Foundation (NSF) in cooperation with the National Research Council of Canada (NRC) and the National Science Council of Taiwan (NSC) and in East Asia by the National Institutes of Natural Sciences (NINS) of Japan in cooperation with the Academia Sinica (AS) in Taiwan. ALMA construc- tion and operations are led on behalf of Europe by ESO, on behalf of North America by the National Radio Astronomy Observatory (NRAO), which is managed by Associated Universities, Inc. (AUI) and on behalf of East Asia by the National Astronomical Observatory of Japan (NAOJ). The Joint ALMA Observatory (JAO) provides the unified leadership and management of the construction, commissioning and operation of ALMA.