1. Proactive Systems Monitoring
Brion Keagle
Assistant Director IT, Core Services
Fitchburg State College
Copyright Brion Keagle This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

2. “If computers get too powerful, we can organize them into committees
“If computers get too powerful, we can organize them into committees. That’ll do them in.” -Author Unknown

3. Event Log Monitoring
Servers will frequently alert the administrator to problems before they turn into incidents. The problem is, usually nobody is listening!
Regulatory requirements (PCI, SOX, HIPAA, etc) may mandate event log review.
A story…

4. The Windows Event Viewer
Provides informational, error, and debugging information via GUI.
Arranged into Application, Security, and System log.
Security log only writable by Local Security Authority Subsystem Service (lsass.exe).

5. The Windows Event Viewer

6. The Windows Event Viewer
Extensible – most 3rd party enterprise software apps and services write information to the Event Log.

7. The Windows Event Viewer

8. The Windows Event Viewer
Drivers and system agents write to the Event Log too:
HP Systems Insight Manager
Dell Open Manage Server Administrator

9. The Windows Event Viewer

10. “The problem with troubleshooting is that trouble shoots back
“The problem with troubleshooting is that trouble shoots back.” -Author Unknown

11. Event Log Monitoring Options
Manual checking – not practical!
Send to syslog server.
NTSyslog open source code
Kiwi Syslog daemon
Home-made scripts, apps.
RSS feed.
Traditional Network Monitoring System (NMS).

12. Event Monitoring Options
We tried this with MOM, but…
By default, MOM forwards nothing. Must set up specific rules to watch for known events.
Rules apply to all servers.
No fine grained control.
Beware – many other NMSs are similarly limited.

13. Event Monitoring Options
Specialized Event Monitoring Applications
Manage Engine Event Log Analyzer
TNT Software’s Event Log Monitor (ELM)
GFI EventsManager
Netikus EventSentry
Others…

14. Event Monitoring Software
What to look for:
Are filters easy to create and manage?
Which Operating Systems supported?
Multiple server and notification groups?
Canned compliance reports?
Receive Syslog notification?
Receive SNMP traps?
Tail a log file?

15. “If you have any trouble sounding condescending, find a Unix user to show you how it’s done.” –Scott Adams

16. Event Monitoring Software
Event Actions
Send
Run script or executable with arguments
Reboot
Stop, start, restart, pause service
IM, RSS
Write to database
Cost?

17. Event Monitoring Software
Other NMS or Configuration Management functionality?
Heartbeat or service monitoring
Performance charts and alerts
Hardware inventory
Software inventory
Environmental monitoring
Cost?

18. EventSentry at FSC Decision criteria: Filters are easy to set up
Many filters are built-in
s from different servers and sources can be sent to the appropriate staff
Robust web reports console
Low cost
Familiarity

19. EventSentry at FSC
EventSentry core functionality: Monitor event logs and forward Warnings and Errors via .
Many other “Actions” (file, syslog, database, print, network message, run script, sound, desktop, SNMP, page, manage processes, shutdown/reboot, Jabber)

21. EventSentry at FSC Events can be targeted to specific admins.
Highly customizable filters
Include Filters
Exclude Filters
Hours
Expirations
Thresholds

22. Results An outage was prevented within 3 hours of installation!
Advance notification has prevented many outages, and shortened many others, as we know about incidents immediately.
Security threats have been mitigated.
Gained more understanding of our environment.
Clarified sysadmin roles

23. “Home computers are being called upon to perform many new functions, including the consumption of homework formerly eaten by the dog.”  ~Doug Larson

24. What to Expect! You will be busy!
Managing high volume of - Only 3 possible circumstances:
Event is benign: Say it’s benign and filter it.
Event is not benign: Fix the problem!
Expect resistance from some admins…

25. What to Expect!
Event Monitoring will expose the “seedy underbelly” of your network!
Rogue admins.
Not following Change Management.
Problems that were “papered over.”
Disparities in workload.
Lack of ownership.
Greater uptime and reliability, happy customers and happy administrators!

26. “Hardware: The parts of a computer that can be kicked.” -Jeff Pesis

27. Questions?

28. What did you think? Your input is important to us!
Click on “Evaluate This Session” on the Mid-Atlantic Regional program page.