MOM Essentials 3: Extending Microsoft Operations Manager (MOM) Part 1 Paul Collins Microsoft UK

Welcome to this TechNet Event FREE fortnightly technical newsletter: “The TechNet Flash” FREE regular technical events hosted across the UK FREE quarterly technical magazine – “TechNet” FREE weekly UK & US led technical webcasts FREE comprehensive technical web site Monthly CD / DVD subscription with the latest technical tools & resources and full-version evaluation and beta software. 30% off until 31 March 2006 We would like to bring your attention to the key elements of the TechNet programme; the central information and community resource for IT professionals in the UK: To subscribe to the newsletter or just to find out more, please visit or speak to a Microsoft representative during the break

Agenda Creating Custom Management Packs Managing Non-Windows Devices with Microsoft Operations Manager (MOM) 2005

Creating Custom Management Packs

Overview MOM 2005 Introduction What’s new for MPs with MOM 2005 State Monitoring Tasks Responses Service Discovery Management Pack Tools

MOM Architectural Overview Data sources – Events: Windows, application, WMI, service change, SNMP traps, timed events, missing events, UNIX syslogs… – Performance data: Used for graphs, reports, and to set thresholds Alerts – MOMs indication of a particular issue  What operators see first – Based on events, performance thresholds or script output Response – Reaction to an alert (auto-resolve, send , page, run script) Management Pack (MP) – Set of Processing Rules to monitor applications – Supporting views and reports

MOM Rule: Unit Of Instruction/Policy Event Rules – Collection rules – Filtering rules – Missing event rules – Consolidation rules – Duplicate Alert Suppression Performance Rules – Measuring – Threshold Alert Rules Rule Provider NT event log Perfmon data WMISNMP Log files Syslog CriteriaResponse AlertScript SNMP trap Pager Task Managed Code File Transfer Where source=DCOM and Event ID=1006 Knowledge Product Knowledge Links to Vendor Company Knowledge Links to Centralised Company knowledge

What Can Management Packs Provide? Monitor line of business applications or business process Monitor the state of your business Monitor third party applications and components Understand how applications are actually being used

What’s New For MPs With MOM 2005? State Monitoring Topology SQL Server Reporting Services Reports Tasks Service Discovery Improved Knowledge

Management Pack Features Alerts: Calls attention to critical events that require administrator intervention – Product Knowledge: Provides guidance for administrators to resolve outstanding alerts Views: Provide targeted drill down details about server health – Performance plots, collections of specific events/alerts, groups of servers, topology, etc. State Monitoring: At a glance view of the state of my servers and applications by server role – Detail to component level Tasks: Enable administrators to investigate and repair issues from the MOM console – Context sensitive diagnostics and remediation Reports: Historical data analytics – Assess operations performance and capacity planning

Health And Diagnostic Modeling Concept What is a Health Model? – Health States – State Transitions: Defined by indicators (e.g., events) Organizes health indicators into an end-user digestible context Alert = actionable health state transition

Health Modeling Process List all Events and Performance Counters Analyze each Event and Performance Threshold – For each define – State Before, State After – Probability – Auto-Retry (self-healing) – “Anti Event” (indicates situation was corrected) – Resolution (action required) Analyze data to define Event and Performance Threshold Groups (e.g., EG1;PG1) Produce Health Model Diagram

State Rules Advantages – State is always current – “What is the server status now?” – Problem taxonomy – What aspect of my server is having the problem? – Role (Exchange, DNS, etc.) – Component (Services, Queues, Mail Flow, Databases) Typical candidates for state-based rules – Numeric thresholds (e.g., perf counters) – Service State

State Terminology

Event Monitoring Event rules can be used for state monitoring An event rule which adjusts state must match at least two event IDs using a regular expression Regular expressions are written in the form 1 | 2 | 3 and wrapped with ^(expression)$ to prevent mismatches

Event Monitoring in Action Rule - Microsoft Operations Manager\Operations Manager 2005\Agents on all MOM roles\The incoming agent queue is full

Performance Monitoring Query and threshold Windows Performance counters as part of your management pack Specify counter attributes to query – Object – Counter – Instance Excellent targets for easy state monitoring

Performance Monitoring in Action Rule - Microsoft Operations Manager\Operations Manager 2005\Agent\Performance Threshold: MOM Service CPU

Other Useful Methods For Creating Custom Management Packs

Management Pack Wizard Ships in the MOM 2005 Resource Kit Build a management pack in 5 clicks containing – Rule Groups – Service Monitoring – Performance Thresholds – Event Monitoring The wizard automatically generate scripts and underlying logic including regular expressions

Management Pack Wizard Advantages Easy to use, requires no real Technical Knowledge Good with any application that writes to the event log and\or has performance counters Automatically creates a service discovery rule Automatically creates a service checking rule with State aware properties

Management Pack Wizard Disadvantages It is dependent on the application writing to the event log and\or performance counters Application needs an Windows service to utilise discovery\service checking rules Event data extracted can be quite raw depending on the application vendor You need to add your own product specific Knowledge Needs to be updated manually when new features or updates are added to the application

Clear Text Log File Monitoring MOM comes with custom App Log provider Gives the ability to read a clear text log file MOM parses each line of log file as a windows event Custom rules can then be created that will search for keywords in the event Ideal when application does not write to event log

Steps for Creating a Clear Text Log Provider 1. Create a Provider: Provider Name: MyApp_Provider Provider Log Type: Generic single line Log Format: Generic Directory: c:\ Pattern: MyAppSampleLogFile*.txt2. 2. Create a Collection Rule Data Provider: MyApp_Provider Store All the Parameter - This will show all the events for the log file 3. Create a Event Rule: Data Provider: MyApp_Provider Criteria: Parameter 4 matches Boolean regular expression '(Error;)' This will alert for the entry which has 'Error;' in the text

SNMP Trap If application is SNMP enabled then MOM can collect SNMP specific data using SNMP WMI Provider SNMP must be set up on Agent Application SNMP MIB must be compiled on MOM agent using SMI2SMIR command Collection rule must be created to get the SNMP traps from application SNMP trap is turned into an event Event rule created to search for specific text

Example SNMP Trap __CLASS=SnmpV1Notification __DERIVATION=SnmpNotification,__ExtrinsicEvent,__Event,__IndicationRelated,__SystemClass __DYNASTY=__SystemClass __GENUS=2 (0x2) __NAMESPACE= __PATH= __PROPERTY_COUNT=7 (0x7) __RELPATH= __SERVER= __SUPERCLASS=SnmpNotification AgentAddress= AgentTransportAddress= AgentTransportProtocol=IP Community=public Identification= TimeStamp= (0x2897D1) VarBindList={ instance of SnmpVarBind { = UPS: Batteries discharged.; }, instance of SnmpVarBind { = 1 (0x1),0 (0x0),0 (0x0),0 (0x0),3 (0x3),0 (0x0),0 (0x0),0 (0x0),6 (0x6),0 (0x0),0 (0x0),0 (0x0),1 (0x1),0 (0x0),0 (0x0),0 (0x0),4 (0x4),0 (0x0),0 (0x0),0 (0x0),1 (0x1),0 (0x0),0 (0x0),0 (0x0),'>' 62 (0x3E),1 (0x1),0 (0x0),0 (0x0);

Missing Event Rule Allows you to alert when an expected event does not occur Ideal for instance where a job is expected to run or a service is expected to start Created in the same way as a standard event based alert rule. Can be used in conjunction with a consolidation rule to look for multiple events

Custom Scripts Can be used to simulate application transactions e.g. remote connectivity over WAN links Health checks on applications to see if essential services are running Collecting information about applications using the registry and WMI namespace Use existing scripts for examples

Creating Custom MP’s

Managing Non-Windows Devices with Microsoft Operations Manager (MOM) 2005

Overview Leveraging infrastructure in MOM – SNMP – Syslogs MOM and Scripts/Managed Code – MOM Scripts – Managed Code Third Parties – Jalasoft – AppMind – Quest Summary

What Can I Monitor? Should be able to monitor anything that is connected and available to MOM How can you get the data/instrumentation out of these different devices/systems and into MOM – Instrumentation (inside out)  SNMP, Syslog – Synthetic transactions (outside in)  MOM + Scripts/Managed Code

The Problem Event Rule Perf Rule Event Perf Data Alert Notification ManagedDevice

SNMP WMI SNMP Provider WMI Event Rule SNMPCollector ManagedDevice WMI Provider SELECT * FROM SnmpNotification Windows MOM

Receiving SNMP SNMP Receiver – Install SNMP and SNMP WMI Provider – Configure SNMP Security – Compile MIB (SMI2SMIR utility) SNMP Sender – Configure community and target MOM – Create event rule(s) with SNMP provider – Deploy rule(s) to SNMP receiver – MOM alert by default is associated to the SNMP Receiver (can change through a script response)

Syslog Application Log Provider SyslogCollector ManagedDevice Event Rule Syslog Port

Receiving Syslogs Sender – Configure Syslog target Receiver – Create event rule(s) with Application Log provider of Syslog type – Deploy rule(s) to Syslog receiver MOM – Rules deployed to agent computer receiving traps and messages – Data contained in description and parameters – Simple string comparison or regular expression – Alert is associated to the IP Address

How is a new computer added? A piece of data is attempted to be inserted into the DB with a new Domain/Computer name – Domain = NTDEV, Computer = MACHINE1 is different to MOM than Domain = BLANK, Computer = MACHINE1 Scenario – If Domain/Computer already exists then the data item is associated to it – Otherwise a new computer is added (Managed Type = UnManaged)

Scripts And Executables Scripts – Script can often collect data and are a very extensible way to insert data into mom (events, perf data, discovery data, alerts) – Programmatically create events and perf data – Don’t create alerts directly (insert events/perf data then use rules to create alerts) Executables – Can be called from a MOM rule – Challenge is getting information back to MOM – Can either write to event log (or other source we can access) or use MCL to go directly MOM

Scripts Script ScriptExecution ManagedDevice Event Rule Timed Provider DataSource Script-generated Data

Creating Events/Performance Data LoggingComputer property on the Event object SourceComputer property on the PerfData object CreateEvent Script Event AgentServer LoggingComputer=DEVICE01LoggingDomain=NonWindows CreateEvent Event CreateEvent Event CreatePerfData PerfData SourceComputer=DEVICE01SourceDomain=NonWindows

Sample Script – ATM Devices Set objEvent = ScriptContext.CreateEvent() objEvent.EventSource = "ATM Error" objEvent.Message = "Insufficient funds available." objEvent.Category = "ATM" objEvent.EventNumber = 232 objEvent.EventType = 1 objEvent.LoggingComputer = "ATM7365" objEvent.LoggingDomain = "ATM" ScriptContext.Submit objEvent

Third Party Extensions Value add is in the knowledge of the non-Windows device May add other Management Pack features – Diagrams – Scripts – Tasks – Reports

Connectors MOM Other Management Product Device Existing monitoring tool might meet requirements Use connector to functionally have a single monitoring environment Might even have simpler solution than a full connector

Third Party Examples CompanyPlatformStrategy AppMindVMS Unix/Linux MCL eXcUnix/Linux Network Devices Storage WMI provider JalasoftUnix/Linux Network Devices MCF, MCL MetilinxUnix/LinuxMCF Quest (Vintela)UnixMCF, MCL

Jalasoft Xian Network Manager

Xian Network Manager 2005 Seamless Integration with Microsoft Operations Manager In depth Monitoring and Management of Network Infrastructure Components Cross Platform Highly Scalable Solution Automatic Scanning / Monitoring for Device Discovery Asynchronous / Real time monitoring Server Linux and Solaris Monitoring Quick n’ Simple Installation and Deployment

Xian / MOM Architecture

Xian / MOM Today Cisco Switches / Routers / PIX / VPN HPProCurve Switches 3COMSwitches NortelSwitches NetScalerSwitches F5 NetworksBig IP APCUPS LinuxRed Hat, SUSE, Fedora Servers SolarisSun Solaris Servers

AppMind System Agent

AppMind System Agent – Features Agent technology for Unix, OpenVMS, Linux and VMWare ESX System Monitoring of CPU, Memory, I/O, Disk etc metrics per OS Process Monitoring of Applications and Daemons Logfile Monitoring of Syslog and Application logs Out-of-the-Box default configuration Failover functionality for redundancy Easily extendable through Scripting C/C++/JAVA APIs

AppMind System Agent – MOM Integration Seamless integration, manage non-Windows systems just like your Windows systems Dynamic integration, systems are automatically discovered and added to MOM Event Rules all with Product Knowledge helping you manage non-Windows systems efficiently Out-of-the-Box Performance View for real-time graphing State View integration with 6 custom Server Roles with 2 – 7 Component each. Nearly all Alerts are Stateful. Diagram Integration for easy graphical overview of all non- Windows systems

AppMind – Roadmap & Purchasing Extended Platform Support: AIX, SCO, Tru64, OpenBSD, FreeBSD, NetBSD and Mac OSX Out-of-the-Box management of Oracle, MySQL, WebSphere, SAP and many other 3rd party applications Evaluation software at

Quest\Vintela VSM

Quest VSM Components VSM Service OpenWBEM Push Installation Update Agent Rule Processor Provider Interface

Quest VSM OpenWBEM ( – Quest is the principal author of this award winning open-source implementation of the CIM specification – VSM’s platform for MOM integration – Open standard – Distributed Management Task Force (dmtf.org) – Event and Numeric Event Providers Other Partners of Quest (VSM) – Does not extend other enterprise management product – Does NOT work without MOM installed

Non-Windows OS Support Linux RedHat AS/ES/WS 2.1 & 3.0 (i386) Linux SuSe 8, 8 Enterprise, 9, & 9.1 Solaris 8, 9 & 10 AIX 5.* HP-UX 11i (11.11 PA RISC)

Management Pack Support Supports: – Computer Groups – Computer Attributes – Rules Groups – Event Rules – Numeric Rules – Performance Data Collection – Automated Responses – Scripting with State Variables – Script API – Reports Management Packs completely supported

MOM VSM Integration

Managing Non-Windows Devices With MOM

Summary MOM is extremely extendable and can be used not only to manage your Microsoft Infrastructure but your third-party apps too – Leverage in the box functionality and Resource Kit Tools – Take advantage of our different partner solutions MOM can be used today to manage your heterogeneous environments – Leverage in the box infrastructure – Take advantage of our different partner solutions