PI Helps Solve NT Server Performance Issues and Network Incidents Or The PI’s Have It.

April 3, 2000Dow Corning Corp. Project Started  October Formal Project Started  Goals  Capture NT Server Performance Data In PI.  Capture Network Performance Data In PI.  Tools  A development PI Server (from Dow Corning)  A development PI License (from OSI Software)  Multiple copies of PI Perfmon Interface (from OSI Software)  A development NetScout License (from NetScout Systems Inc.)

April 3, 2000Dow Corning Corp. What NT Performance Data Is Being Captured in PI?  NT Performance Counters through the PI Perfmon interface.  We have had it running since May 17, 1999 in an un-official use as an aid to Jon Peterson

April 3, 2000Dow Corning Corp. Early Findings  Memory on our PI Servers was eroding. PI Trend Available. PI Batch was the main problem.

April 3, 2000Dow Corning Corp. PI-Batch Process Memory Leak

April 3, 2000Dow Corning Corp. Early Findings  Memory on our PI Servers was eroding. PI Trend Available. PI Batch was the main problem.  PI came out with a fix for PI Batch. (Good Job Jon.)  PINetManager had a leak.

April 3, 2000Dow Corning Corp. PINetMgr had a leak.

April 3, 2000Dow Corning Corp. Early Findings  Memory on our PI Servers was eroding. PI Trend Available. PI Batch was the main problem.  PI came out with a fix for PI Batch.  PINetManager had a leak.  PI fixed the leak.

April 3, 2000Dow Corning Corp. Later Findings  PI when run on NT 4.0 SP5 also fixed a leak in PIArchss.

April 3, 2000Dow Corning Corp. PIArchss Process Before SP5

April 3, 2000Dow Corning Corp. Later Findings  PI when run on NT 4.0 SP5 also fixed a leak in PIArchss.  Ensign Agents (another tool used in Dow Corning) can go nuts and lock up one of our NT MSSQL Servers

April 3, 2000Dow Corning Corp. Ensign Agents

April 3, 2000Dow Corning Corp. Later Findings  PI when run on NT 4.0 SP5 also fixed a leak in PIArchss.  Ensign Agents (another tool used in Dow Corning) can go nuts and lock up one of our NT MSSQL Servers  Exchange Server Lost it’s STORE process.

April 3, 2000Dow Corning Corp. Exchange Server Notice the LOSS then change of STORE process from Exchange

April 3, 2000Dow Corning Corp. NT Systems Currently Being Watched!  PI Server  MSSQL Servers  Exele's – EdictVB Server  FOXBoro's NT version (Our Control System Support Group is thrilled.)  Intellution SCADA servers (Our Control System Support Group is thrilled.)  Automatic Data Collection ADC Server  Exchange Servers

April 3, 2000Dow Corning Corp. Locations Being Monitored  Midland – Michigan USA  Barry Wales - UK  Chiba - Japan

April 3, 2000Dow Corning Corp. What Network Data Is Being Captured In PI?  Started in October  NetScout can be configured to use MSSQL.  We added two tables to the MSSQL to create an association between NetScout counters and PI Tags.  Basic Network data is captured from our Global WAN.  We are using 1 minute summarized data from NetScout.

April 3, 2000Dow Corning Corp. Impact of Network on Exchange Server Exchange Server ->

April 3, 2000Dow Corning Corp. Network Benefits of PI Data  Spotted several instances of Over Utilization (Can almost predict instances of Network outages for users.)  Helped to determine a Baseline to determine the need for purchased band width and line speed. ($ can be found here.)

April 3, 2000Dow Corning Corp. In Summary  We helped OSI touch up some of it's code.  We spotted some troublesome software from other vendors.  We help to trouble shoot NT Server issues and outages.  We help to baseline our Global Network.  We saved some $.

April 3, 2000Dow Corning Corp. How Did We Do It?  PI Tag Configurations for PI Perfmon  PI Perfmon Interface  SQL Server Use for NetScout  PI Tag Configurations for NetScout  Lessons Learned

April 3, 2000Dow Corning Corp. Lessons Learned 1  Run PI Perfmon on the server it is monitoring.  This uses the PIAPI and Buffering to push the data to a PI Server.  Trying to PULL data uses to much network bandwidth.  Trying to PULL data leaves you vulnerable to Network Outages.  Remember we are doing this across a wide area network.

April 3, 2000Dow Corning Corp. Lessons Learned 2  Really think through the compression and exception deviation percentages.  Memory usage varies a LOT.  CPU usage varies Even More.  Regression Analysis ProcessBook Addin can save you a lot of stats time.

April 3, 2000Dow Corning Corp. Lessons Learned 3  You CAN use SPC tools on an NT Server Process.  NT Server Process should run acceptablely under the rules of SPC.  You can find out what impacts you and sends you out of control.  Alarming based on Out Of Control can be done.  Alarming based on Process Conditions is also possible.

April 3, 2000Dow Corning Corp. Lessons Learned 4  OSI Believes in Continuous Improvement.  PI Batch Improvement  PI NetMgr Process  PI Archss Process

April 3, 2000Dow Corning Corp. Lessons Learned 5  When It comes to Network Stats, It is hard to tell when you have enough.  NetScout Probes Offers a lot of data. At this point Dow Corning is only capturing the tip of the iceberg.  OSI is working with NetSout to get at much more of the data.

April 3, 2000Dow Corning Corp. Lessons Learned 6  Watch Data in Long and Short Views.  Some items have data that takes weeks to develope.  Some items are best seen in the second they occur.  Only a long term historian can give you this long and short view of events.

Get the Data Have Some Fun