Unicenter Automatic Duplicate Suppression EvtADS -EvtAds r11 -Field Developed Utility -January 17 th 2006.

Unicenter Automatic Duplicate Suppression EvtADS -EvtAds r11 -Field Developed Utility -January 17 th 2006

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 2 Disclaimer This is a field developed utility targeted to be included in the future release for NSM. It is not supported by the CA Support organisation.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 3 Acknowledgments -Kevin Higgins - Cap Gemini UK plc

Introduction

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 5 What is EvtAds? -A simple but function rich utility to intelligently manage duplicate events to aid in auto-ticketing and eliminate opening multiple problem tickets for the same problem. -Automatic Duplicate Suppression -Tool for Windows Event Management

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 6 Problem Summary -Unicenter message processing repeats messages on the operator console unless explicitly discarded. -Such duplication may prevent use of auto-ticketing or occasionally cause acute message storms, (where too many messages per second are generated). -Message storm can result into denial of service if extensive instrumentation is carried out to address duplicates.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 7 Automatic Duplication Suppression -EvtAds is Unicenter Event Management Anti- Duplication Suppression driven by Event Message Actions. -It is specifically designed for scalability to handle large message storms without any performance degradation. -In a lab environment, it has successfully processed message storm of over a million events

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 8 Automatic Duplication Suppression -ADS is called using EXTERNAL Message action and determines if the event has been previously seen within the reset interval. It returns with an &rc indicating the number of times an event / alert has been previously seen. -Analysing &RC value assists in auto-ticketing and prevents multiple tickets been raised for the same problem. -Provides means of escalation based on the number of alerts received for the same problem. For example, if 15 failure events are received within reset interval period for the same problem, it would make sense to escalate the problem to emphasis the impact of the problem. Without ADS, this may result in 15 problem tickets being raised

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 9 Why EvtAds? -Unicenter Event Management provides frequency based message rules. This is based on Counter and Interval. -Frequency based event rules are driven by message records and may require complex rules to integrate with Help desk. -Interval computation for Frequency event rule is based on time the first event occurred. EvtAds works on interval based on last update time – this is key for effective duplicate suppression.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 10 Help Desk -To provide integrity, it is important that duplicate problem tickets are not raised. Multiple problem tickets for the same problem can easily distract the main focus of resolving the problem. -For this reason, duplicate Alerts should be identified and discarded (or in some cases escalated). -If duplicated alerts exceed an acceptable limit, the problem should be escalated to minimise the impact. It is important to identify the number of times an alert has occurred within a clear down interval.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 12 EvtAds Functions -CAds -Checks for duplicate Events -CVersion - Displays Version details -CStats -Displays how many events are cached -CList -Generate list of cached entries for review

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 14 ClearDownInterval -This is used for computing if the cached event entry is current and not expired. It is based on Last Update time of the event. -If (Last Update Time + ClearDownInterval) < Current Time, the cached event entry is expired and the duplication count of the cached event is reset to zero. -This enables identification of new occurrences of problems outside the ClearDownInterval

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 15 define msgact name=(*,10) action="EXTERNAL" attrib="DEFAULT" color="DEFAULT" condop=" " evaluate='Y' quiet='N' status="ACTIVE" sim='N' text="EvtAds.DLL CAds /T:120 /M:""&text""" EvtAds Message Action CAds Function Function Name ClearDownInterval Message Text Action

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 16 define msgact name=(*,10) action="EXTERNAL" attrib="DEFAULT" color="DEFAULT" condop=" " evaluate='Y' quiet='N' status="ACTIVE" sim='N' text="EvtAds.DLL CAds /Q /T:120 /M:""&text""" EvtAds Message Action CAds Function Query Option Function Name ClearDownInterval Message Text Action Query Option

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 17 Query Option -Query option function works in the same as CAds except the cache count is not updated. The clear down rules still applies. -This function is extremely useful when conditional logic is required such as “Stop forward action if more than 10 forwards have failed due to destination down”.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 18 Sample Application of Query Option There is another MRA for %CAOP_E_504 with EXTERNAL action. This issues CAds Function call with /M”&1,&6”. This updates the cache counter to identify the number of times, the forward has failed within cleardown interval. Prior to forward action, Query option is requested to determine the number of times %CAOP_E_504 message has been generated. The query option will not update the counter. In this example if more than 5 FORWARD to node BREER01PACO have failed, forward action is suspended with appropriate messages. This would be good candidate for EVALUATE function

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 20 define msgact name=(*,10) action="EXTERNAL" attrib="DEFAULT" color="DEFAULT" condop=" " evaluate='Y' quiet='N' status="ACTIVE" sim='N' text="EvtAds.DLL CAds /T:600 /F:2 /M:""&(2:9)""" EvtAds Message Action Search & Replace Override Function Name ClearDownInterval Message Text Search & Replace Item

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 21 CVersion -Display EvtAds Version details define msgrec msgid="EVTADS CVersion" type="MSG" msgnode="*" desc="What version of EvtAds are you running" cont='N' msgact='Y' wcsingle='?' wcmany='*' case="y" regexp="n" define msgact name=(*,10) action="EXTERNAL" attrib="DEFAULT" color="DEFAULT" condop=" " evaluate='Y' quiet='N' status="ACTIVE" sim='N' text="EvtAds.DLL CVersion"

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 23 define msgrec msgid="EVTADS CStats" type="MSG" msgnode="*" desc="Tell me how many entries have been cached so far" cont='N' msgact='Y' wcsingle='?' wcmany='*' case="y" regexp="n" define msgact name=(*,10) action="EXTERNAL" attrib="DEFAULT" color="DEFAULT" condop=" " evaluate='Y' quiet='N' status="ACTIVE" sim='N' text="EvtAds.DLL CStats" CStats -Display number of events cached

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 24 CStats CStats - 2000 entries cached

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 25 define msgrec msgid="EVTADS CList" type="MSG" msgnode="*" desc="Dump Current Cached entries to a ascii file" cont='N' msgact='Y' wcsingle='?' wcmany='*' case="y" regexp="n" define msgact name=(*,10) action="EXTERNAL" attrib="DEFAULT" color="DEFAULT" condop=" " evaluate='Y' quiet='N' status="ACTIVE" sim='N' text="EvtAds.DLL CList" CList -Dumps cached entries for review -List file placed in EvtAds directory. -File name -yyyymmdd.ads

Clear Down Interval

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 29 Clear Down Interval -Clear Down Interval is Reset Interval. Can be different for different message rules. -Cached entry is expired if the last update of the event is older than clear down interval. -If no Clear Down Interval specified on the message rule, it defaults to Initialisation Option default

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 31 Hierarchy -/T: override on message action -Initialisation Default from Ini File

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 34 What is Purge? -Remove matured cached event entries from the queue -Cache entries eligible for Purge if Current Time > (LastUpdateTime + (ClearDownInterval * PurgeLag) -PurgeLag is customisable option via configuration GUI.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 36 Why Purge? -This is a simple housekeeping function to keep the Queue to a manageable size. -In the lab environment, Queue Size of 300,000+ has been tested successfully. -This allows removal of single event entries with no probability of reoccurrence

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 37 How is Purge Initiated -Purge can be initiated on demand or automatically performed at the specified Check Interval -CheckInterval = nn -where nn = interval value in minutes. -INI File option - configured via Configuration GUI -Check Interval value should be based on the Event Activity. In most cases Check Interval of 2hours should suffice. However, in lab environment, this has worked fine with value of 1 minute.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 39 Purge On Demand -To run purge on demand, simply call COptions – Purge function. -Utility is built with threshold monitoring of Queue Size. If it exceeds Warning or Critical thresholds, it generates Warning or Critical Messages. -This messages should be reviewed with the view to execute Purge On Demand request on receipt threshold violation messages

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 41 Automatic Purge This shows Purge is executed every 15 minutes. CheckInterval value set to 15

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 42 How to disable automatic Purge -If you wish to stop the automatic purge process and certainly not recommended, then this can be accomplished by manually updating EvtAds.dat file. -Set CheckInterval=0 -This will disable to automatic purge process -The Configuration Wizard, will not permit this value to be set to 0 as it is not recommended

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 43 Purge 12,000+ Entries purged in no time

Search and Replace

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 45 Why? -Some events may include fields such as Time which would make the event unique and thus potentially not suitable for ADS. -If fields such as Time are not relevant to determine the duplication count, then search and replace function should be reviewed.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 46 How? -Using Configuration GUI, specify POSIX regular expression to search the message text passed to EvtAds and replace with the supplied text. -GUI provides a TEST function to verify your search and replace function. -Each Search & Replace item is allocated a sequential number. This number should be specified on the CAds function as /F:n option, where n is the item number

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 47 Replace Time Field with hh.mm.ss -Define Search and Replace Item. -Item number for this field is 2 -ReformatEventSearch Replace Item Search RegExp

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 48 Search & Replace Override Option Item Number

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 49 Time Field Replaced with hh:mm:ss cawto EVTADS Jobxxxx failed at 10:11:38 cawto EVTADS Jobxxxx failed at 10:11:39 cawto EVTADS Jobxxxx failed at 10:11:40 cawto EVTADS Jobxxxx failed at 10:11:41 cawto EVTADS Jobxxxx failed at 10:11:42 Generate Time Based Events Cached Entry

Message Prefix

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 53 Message Prefix -Some Message prefix are not suitable for EvtADS suppression. For example you may want to see all %CAOP messages or messages generated by EvtADS itself. -This function provides assurances in case catch all message record rule is used for ADS, such as *

Cache Statistics

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 55 Release r11 -r11 release has been enhanced to maintain some degree of statistics of the cache events. This enhances the CList function if MaintainStats option is set to 1. -By default, it will work as prior releases i.e no history of cached events will be maintained

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 56 MaintainStats -This option is not currently configurable from the wizard. If the option is set to 1, it will enable Stats option

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 57 MaintainStats – Automatic Mode -If MaintainStats option is enabled, it will maintain cache count (duplicate count) of each cached entry. -CheckInterval, is a configurable option from the configuration wizard. It specifies how often to perform housekeeping on the expired cache entries i.e entries that gone past (CleardownInterval * PurgeLag). -If MaintainStats is enabled, at each CheckInterval, it will automatically generate statistics and append to the file YYYYMMDD_ads.csv.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 58 MaintainStats – Manual Mode -If MaintainStats option is enabled, stats information can also be manually requested by calling function CList.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 59 define msgrec msgid="EVTADS CList" type="MSG" msgnode="*" desc="Dump Current Cached entries to csv file if MaintainStats enabled" cont='N' msgact='Y' wcsingle='?' wcmany='*' case="y" regexp="n" define msgact name=(*,10) action="EXTERNAL" attrib="DEFAULT" color="DEFAULT" condop=" " evaluate='Y' quiet='N' status="ACTIVE" sim='N' text="EvtAds.DLL CList" MaintainStats - CList -Dumps cached entries for review -List file placed in EvtAds directory. -File name -yyyymmdd_ads.csv

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 60 Stats File This shows EvtAds was processing 1700 events per seconds

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 61 MaintainStats -Id -Groups all entries for a specific stats request -Cache Rate -Number of Events cached per second for the respective message text -If the entry was purged due to no activity, the event rate will be calculated from the time the entry was re-cached till the lastupdate time. If the entry has not been previously purged, then the event rate is calculated from the time the entry was first cached till last update time.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 62 MaintainStats -Total Cached -Total number of Events cached for the respective message text -If the entry was purged due to no activity, the “Total Cached” count is from the time the entry was re-cached. If the entry has not been previously purged, then the “Total Cached” count is from the time the entry was first cached.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 63 MaintainStats -DuplicateCount -The total number duplicate entries within the current cleardownInterval period. - ElapsedSeconds -The number of seconds from the time the cache entry was created or recreated if previously purged -Request -Request Type. The entries written to the csv file was as a result of CheckInterval, then this is set to Auto. If the CList function is called by the user, this is set to Manual -FirstTimeStamp -The timestamp of the first occurrence of the cached message text. If the entry was previously purged, it is timestamp when the entry was re-created

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 64 MaintainStats -LastUpdateTimestamp -The timestamp of the last cached entry update

Scalability

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 66 Test Plan -Generate Message Storm of quarter million (250,000) events -Analyze the latency with EvtAds caching the entries from this message storm -Display the first entry and last few entries -Re-run the tests without caching entries. -Analyze the latency without EvtAds caching the entries

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 67 Message Actions 1.Cache Entries based on message token 2 and 3. 2.Check if first entry. If so, display the text 3.Check if the last few entries. If so, display the text 4.Otherwise do not display the message on the Unicenter Console

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 68 Generate Message storm -Message Storm Ended at 08:20:29

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 69 Message Storm 250,000 Events -250,000 Events message storm cached by EvtADS -Latency on the last cached event 71 seconds

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 70 EvtAds This shows EvtADS processing 2121 events per second

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 71 Re-run test without EvtADS -Re-run the tests without executing the 4 MRA processed by the message storm The latency without EvtADS caching 250,000 events and not processing 4 MRA is 50 seconds. That is 21 seconds less than EvtADS

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 72 Purge 60,000+ Entries purged under a second

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 73 Cache Table Search Time -If large number of entries are cached (over 50,000), degradation in response time should be expected as the time to search for a specific cache entry in the cache table will increase as well -EvtADS monitors its cache table and generates alert messages if it exceeds Warning or Critical threshold. -If you are planning to trigger EVTAds with MRA such as “*” i.e for all messages, then please review purge housekeeping options such as PurgeLag and CheckInterval to maintain the cache table to a reasonable size

Install

Configuration Wizard

Verification

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 85 Verification -Execute EvtAds_Demo.cmd few times and verify -Duplicates Message Discarded Test -Only one message should displayed evtAds_001 New Entry Cached - EVTADS DupTest This should only show the first message -Search and Replace Function -Only one message displayed for duplicate Time field messages evtAds_001 New Entry Cached - EVTADS Jobxxxx failed at 10:11:38 -Escalation -3 Message displayed -Normal when duplication 0 -Warning with 10 th occurrence of message -Critical with 15 th occurrence of message

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 86 Alert based on 10 th and 15 th occurrences Count based Escalation verification. Warning EVTADS ServiceDesk Process displayed on 10 th occurrence of the duplicate message within 2 minutes interval. Critical message on 15 th occurrence

EvtADS Initialize

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 88 Message Storm at Startup -If the message storm is generated and EvtADS is not initialized, then there may be a small delay at start up. This delay is mainly due to CA OPR daemon has to load the EvtADS external library. -For release r11, it is recommended to initialize EvtADS at startup of CA OPR daemon, so that external library is pre-loaded.

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 89 How to initialize EvtADS at Startup? -Define Message Records Actions for the Message Id -%CAOP_I_DAEMONINIT, Console Daemon initialized on node -Define Message Actions as defined in the file EvtADS_init.cautil

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 91 Initialize EvtADS at Startup -If EvtADS is not initialized at startup, CA OPR daemon will initialize the first execution or EXTERNAL message action for EvtADS library

© 2005 Computer Associates International, Inc. (CA). All trademarks, trade names, services marks and logos referenced herein belong to their respective companies. 92 Questions and Answers Any questions?

Unicenter Automatic Duplicate Suppression EvtADS -EvtAds r11 -Field Developed Utility -January 17 th 2006.

Similar presentations

Presentation on theme: "Unicenter Automatic Duplicate Suppression EvtADS -EvtAds r11 -Field Developed Utility -January 17 th 2006."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Unicenter Automatic Duplicate Suppression EvtADS -EvtAds r11 -Field Developed Utility -January 17 th 2006.

Similar presentations

Presentation on theme: "Unicenter Automatic Duplicate Suppression EvtADS -EvtAds r11 -Field Developed Utility -January 17 th 2006."— Presentation transcript:

Similar presentations

About project

Feedback