New SA Training Topic 9: Logging, Monitoring, and Performance  Logging  Windows – “Auditing”  Linux – syslog  Monitoring  MRTG  Big Brother  Performance  Perfmon  Sysstat

Logging Windows Logging is usually dealt with as “Auditing”, this information is reviewable in the Event Viewer Many items are NOT audited by default, this must be enabled Logging is often incomplete as compared to that done by Linux IIS logs to %systemdir%\system32\LogFiles\ These can be set to store elsewhere though The content and form is to some degree configurable

Logging (cont.) Linux Syslog – syslogd collects messages from processes and routes them as needed Syslog.conf entry format facility.levelaction Facilities – kern, user, lpr, daemon, auth, authpriv, mail, cron, syslog, mark, local* (0-7) Levels – emerg, alert, crit, err, warning, notice, info, debug, none, mark (selected or higher) Action – write to file (often /var/log/messages), message a user (or list of users), syslog on another host etc.

Logging (cont.) # Sample syslog.conf – This prints most sys. events to the console, # emergencies to everyone, alerts to root, and auth.info and all # warnings to otherhost kern.warn;*.err;authpriv.none /dev/console *.emerg * *.alertroot # send mail and kernel/firewall messages to their respective logfiles mail.* /var/log/mail kern.* /var/log/kernel_n_firewall # operators: “=“ only this, “!=“ all but this, “!” log levels below kern.=alert/var/log/kernel_alerts # save the rest in one file, but exclude mail from these (.none) *.*;mail.none /var/log/messages

Monitoring A service is not in production until it's monitored Level 1 - would include things like viewing Windows processes, Linux “top” command, netstat (both platforms), etc. Level 2 - might be packet sniffers such as tcpdump and Ethereal Level 3 - might included SNMP based utilities You should already be familiar with the first two levels

MRTG Multi Router Traffic Grapher (MRTG) A tool to monitor the traffic load on network-links Runs from cron (Windows - scheduler?) Generates HTML pages and images that provide a LIVE visual representation of this traffic Based on Perl and C and works under UNIX and Windows NT RRDtool New product by same author Improved data consolidation and graphing Needs additional software to collect data (Cricket)

MRTG (cont.) GREEN ###Incoming Traffic in Bits per Second BLUE ###Outgoing Traffic in Bits per Second

MRTG (cont.) GREEN ###Incoming Traffic in Bits per Second BLUE ###Outgoing Traffic in Bits per Second

Big Brother Monitors System and Network-delivered services for availability An almost real-time indication of network status is displayed on a color-coded web page Can handle notification via , pager, or text messaging

Big Brother (cont.)

Performance Performance is a huge topic Three step cyclic process of managing performance Measure - determine current performance levels Estimate - required/best case performance levels Tune system - to meet requirements/best case levels

Perfmon Lets you keep an eye on just about anything Things are grouped into “objects” and objects are divided into “counters” Example: things related to the CPUs are in an object called “processor” Lets you either log info long-term or view in real time Start/run/perfmon Start/Programs/Admin Tools/Performance

Perfmon (cont.)

Counter logs Create a log based on Objects and/or Counters (same items viewable in Performance Monitor) Store the collected information for later viewing and evaluation Trace logs Event Tracing for Windows (ETW) is a tool for performance- testing and diagnostics Gives developers a mechanism with which to determine their applications' performance effects on Windows Server 2003, Windows XP, and Windows 2000 platforms Administrators can use ETW to find out what's happening in their internal Windows systems, Microsoft applications (e.g., Microsoft IIS), and third-party applications and troubleshoot any problems they might find ETW can also help administrators with capacity planning by letting them monitor a system under real workloads to see how it performs for a given set of transactions

Perfmon (cont.) Alerts Give notice when “something” happens Can alert by Adding log entries Network messaging Running a program Some examples free disk space (logical disk/free megabytes) general network congestion (network percent network utilization) – requires Monitor Agent logon attempts for ftp or http servers logon errors (Server/errors logon)

sysstat A set of commands for Linux sar - collects and reports system activity information The information collected by sar can be saved in a file in a binary format for future inspection The statistics reported include I/O transfer rates, paging activity, process-related activities, interrupts, network activity, memory and swap space utilization, CPU utilization, kernel activities and TTY statistics, etc. sadf - used to display data collected by sar in various formats (XML, database-friendly, etc.) iostat - reports CPU utilization and I/O statistics for disks mpstat - reports global and per-processor statistics Both single and mulit-processor machines are fully supported

sysstat (cont.) Apple’s Dashboard Widget for Sysstat