Logging and Monitoring

Motivation Attacks are common (see David's talk) – Sophisticated – hard to reveal, (still) quite limited in our environment – Frequent “noise” Clouds make the situation more complex – You address issues caused by random users

Motivation Goals – Efficient IR – Build and maintain good reputation Means – Prevent incidents from happening – Timely response – Efficient handling of incidents Automation of procedures, reporting

Revealing Incidents - Motivation Detecting attack from own monitoring – EGI “FedCloud” incident at CESNET – Network monitoring revealed dDoS attacks from a cloud machine – Further forensics showed compromised Tomcat account (with enabled default password) Report from outside – “We found malicious activity originating from your IP address during ”

Attackers Behavior - Motivation Common scenario – Get user access to a machine via common vectors – (escalate privileges) – Hide traces – Launch a “bot”,... Forensics tries to identify the steps

Setting up Basic Elements Logging and monitoring is crucial for incident response and prevention Consider involvement of cloud users infrastructure vs. vms Basic tools – logs – patch management – VM monitoring – Network monitoring

Log Management

Gathering Logs Make sure key components generate logs Verify that logs are complete – IP, usernames – traceability Think about policies Collect logs centrally (syslog) – Attackers wipe logs – Easy evaluation Provide support for VMs – Pre-configured images, documentations, …

Processing Logs Check logs automatically – Logwatch, … – Identify suspicious patterns – successful ssh attacks Logs may be large – Prepare a proper solution (ELK) – Identify what needs to be retained (policies) – Prepare storage

Patch Monitoring

Patch Monitoring using Pakiti Patch monitoring, detects known vulnerabilities Client - server architecture In production use by EGI CSIRT, Nagios probe against WNs

Pakiti in EGI New vulnerabilities assessed EGI CSIRT / SVG → Critical or High-rated. Sites are requested to address the issue, Update package(s) and/or apply mitigations. 7 bussiness days for new vulnerabilities, 2 days for re-occurences Sites failing to address the issue may be suspended Results available to sites/NGIs (limited access)

Pakiti GUI

Patch Management in EGI

Monitoring of Cloud Machines be faster then attackers

VM Assessment Users are creative Identify common attack vectors – Default/weak passwords – Testing accounts Develop procedures and policies – Regular audits Integrate with general issue handling – automation

VM Assessment at CESNET Regular network scans of cloud machines – SSH accounts – Tomcat accounts – open “amplifiers” Policies being updated – VMs are contained after found vulnerable

Network Monitoring

Why Monitor Network? Everybody leaves traces in network traffic (you can’t hide). – Identification of attack attempts – Identify successful attacks – Incident analysis Outcomes improve knowledge about network

Monitoring using Flows Passive monitoring collecting metadata – No support needed from users/customers Flow contains key information about every connection – Source, destination IP, ports, times, protocols, … – No content data (mostly) Very valuable for forensics and attack analysis

Collecting Flows Several solutions available "NetFlow Architecture 2012" by Amp 32 (wikipedia.org)

Using Network Monitoring Data Automated processing – Successful phishing, network attacks, suspicions IPs, … Auxiliary processing

Questions ?