4.3.4 OPC Historical Data Access Industrial Automation Automation Industrielle Industrielle Automation 4 Access to devices 4.3.4 OPC Historical Data Access Prof. Dr. H. Kirrmann ABB Research Centre, Baden, Switzerland 2012 May, HK

HDA: Overview OPC Common Overview: usage and specifications OPC as an integration tool Clients and Servers: configuration OPC Technology, client and custom interface OPC Data Access Overview: browsing the server Objects, types and properties Communication model Simple Programming Example Standard and components OPC Alarms and Events Specification Overview: definitions and objects Events Alarm Conditions Automation Interface OPC Historical Data Specification Overview

Historians: An introduction The role of HDA will become clearer after a first glance at historians The process database decouples the plant from the analysis tools. It contains recent and historical data. Invensys (Wonderware) Archestra

GE Fanuc/Intellution iHistorian (iFix...) Historians Example GE Fanuc/Intellution iHistorian (iFix...) Questions to the historian: What was the value of FIC101 last week ? What was the flow average during October ? Which were the daily averages in October ? What was the total flow in each month ? How much fuel did we use for the batch ? Give the answer in form of tables, pie diagrams, spreadsheet, reports Features: Unlimited Point Collection Sub-Second Data Collection Rates Enhanced Data Compression True Thin Client Administration Fault Tolerant Architecture data collection, archiving and retrieval report generation computations (e.g. VBScript) secure access (FDA 21 CFR 11) 20'000 actions/ s, up to 100'000 data points

ABB’s Information Manager Applications Calculations Common Access API (OPC) Real-Time Data Base Soft Point Server Aspect Server Directory Generic OPC Servers Connectivity Aspects PLC’s and other devices Remote Controllers

2003 Global Information Management Market by Industry Historians: Market 2003 Global Information Management Market by Industry 11% 2% 21% 1% Chemical Food and Beverage 6% Petroleum Pulp and Paper 13% Pharmaceuticals Power 13% Textiles Waste and Water 12% Other 21%

AIP/PPA (embedded) Historian is another product (small player) Historians: Players ABB Information Manager (formerly Enterprise Historian, bases on Oracle) AIP/PPA (embedded) Historian is another product (small player) AspenTech InfoPlus.21 Complete offering with wide range of accompanying applications and services Very good user interface OSISoft PI 3.4 Largest installed base (perceived as de facto standard) Easy-to-use Perceived low cost (PI 3.4: 1 Mio data points / 80'000 /s) GE-Fanuc Intellution iHistorian Relatively new to market. Product created with Mountain Systems Causing a stir for OSIsoft, AspenTech on pricing Some small players Honeywell (POMS) Uniformance Invensys (Wonderware) IndustrialSQL 8.0 Rockwell Automation (RSBizWare) Historian Siemens (WinCC) Historian (bases on MS SQL Server)

Historian implementation application OPC HDA ODBC/SQL and OPC HDA are complementary log organisation "SQL" history database collector operator actions corrections "OPC" field data

HDA: Historical Data Access e.g. Trend Analysis e.g. Event Logger HDA Clients independent processes OPC HDA Server hidden calculations history database raw and ordered data collector OPC DA Server proprietary data acquisition Field device Field device

HDA: Purpose An OPC HDA server gives access to a historical data base (logs) in which data from the process have been collected and time-stamped, possibly through an OPC DA interface. The OPC HDA interface clients, such as trend analysis, product tracking or data mining, that require ordered access these data logs. The OPC HDA interface allows to: - browse the historical data base - retrieve data through proper filtering, e.g. by date range, by identity, by property - build aggregates over the retrieved data, such as average, minimum, maximum. - enter new entries, correct entries or remove entries - enter / delete annotations in the history data base

Data in the historical database are identified by their HDA: Raw log 12.3.02 13:40 Gpcpt2ofpbonne 4824 By definition, values are registered when they change. (even if data are acquired by periodic polling) 12.3.02 13:40 Cpt2bac 50 12.3.02 13:40 Gpcpt2bac 70 12.3.02 13:40 Gpcptbe2 45 12.3.02 13:41 Gpcpt1bac 151 12.3.02 13:41 Gpcpt1ofpbonne 4826 Data in the historical database are identified by their itemID (here, represented by their name), value, (of the respective type) quality (good, stale, bad), and timestamp (UTC). 12.3.02 13:41 Gpcptae2 45 12.3.02 13:41 Cpt1bac 49 12.3.02 13:41 Gpdefr2 64 12.3.02 13:41 Gpvoydef 2 12.3.02 13:41 Gpr3tempscycleprd 318 12.3.02 13:42 Gpstn1e1 16 12.3.02 13:42 Gpalarme1 12.3.02 13:42 Gpalarme2 12.3.02 13:43 Gpetatmodemarche 2 12.3.02 13:43 Gptpscycle 1346 12.3.02 13:43 Gpetatmodemarche 1 12.3.02 13:43 Gpdefgene1 16 12.3.02 13:43 Gpetatmodemarche 12.3.02 13:43 Gptpscycle 317 12.3.02 13:43 Gpdefr2 12.3.02 13:43 Gpvoydef 12.3.02 13:43 Gpdefgene1 12.3.02 13:44 Gpetatmodemarche 1 12.3.02 13:44 Gpr2tempscycleprd 1992 12.3.02 13:44 Gptpscycle 435 12.3.02 13:44 Gpalarme3 1 12.3.02 13:44 Gpalarme4 1 12.3.02 13:44 Gpalarme3 12.3.02 13:44 Gpcpt2ofpbonne 4823

HDA: How to reduce raw data (even before HDA comes into play) Tag values are sent to the OPC DA server when they change (on events), the collector records these data in a log that is organized as a circular buffer. Since a log has a limited storage capacity, data are reduced by several means: 1) if a variable is received more often than the log's minimal storage interval (e.g. 1s), the log keeps the latest of all values of the interval 2) if (and only if) a received variable changed by more than the "exception deviation" (e.g. 5%, analog values only), it is entered into the log. 3) if a variable changed by less than the exception deviation, it may be forced into the log after the log's maximum storage interval (e.g. 4 s) elapsed. min time max time max time max time max exception deviation min time = process value (event) = log entry Two situations can occur: - the time scale of the log is smaller than that of the trend display in this case, values have to be interpolated to be displayed correctly

HDA Application: Trend Display Parameters: time scale (with possible offset, zoom, pan) amplitude scale (low range, high range, scale units) style: smoothed, stepped, filled (several ways to display the same data) extrapolate: how to display values not received (e.g. because they did not yet change) log: how were data sampled

A log registers only one data point ! HDA: More logs A log registers only one data point ! (log templates can be used for several datapoints) log 2.1 log 2.2 log 1 log 2 Data in the log may be: Measured values (original, from the OPC DA) Calculated values Corrected values (replaced or inserted) Annotations (explanation strings) raw log Collector (OPC DA Client) other servers OPC DA Server XML / CSV Field device Field device

HDA: Hierarchical logs long-term log 1 hour_forever  5.2 MB / Year 1 2 3 27 28 29 30 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 september octobre october 1 mn_7 days log  6 MB yesterday d-2 d-3 d-4 d-5 d-6 d-7 today 1s_24 hours log: 50 traces @ 12 B  52 MB 23 24 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 A hierarchical log is built on the data contained in the parent log. To reduce the log size, several aggregations can be applied: - record only maximum, minimum, average over a period, etc... Actual data

HDA role An HDA server allows to browse the logs and filter the logs to be read. It can filter them according to their attributes, such as name of the datapoints, type of the datapoint, engineering units,.... An HDA server then selects the logs to be read. An HDA can then read the logs, defining a certain period (Start time, End Time, and the maximum number of entries to be retrieved. When reading the log, the HDA server can process the log by applying aggregations The "min time", "deviation", "max time" rules used to collect raw data is also used as an aggregation to build new logs. Therefore, hierarchical logs can be built out of a raw log by an OPC HDA service. Do not confound aggregates and attributes.....

HDA: Server The client can query the HDA server for its capabilities, and in particular: - which LocaleID (national languages) the server supports (as in DA and AE servers) - which attributes the server supports (the attributes of its items is a subset thereof) - which aggregates the server support (and the client can request) - if the server supports correction / deletion of data (synchronous and/or asynchronous) - if the server supports insertion of annotations (synchronous and/or asynchronous) Many capabilities of the HDA server are optional.

HDA: Browsing: structure Like the DA and AE interfaces, OPC HDA server offers a browser with methods allowing to navigate up and down, and to retrieve branches and leaves at each position. If filters are defined at its creation time, an HDA browser only browse for items with specific attributes (e.g. Engineering Units = m/s). The filtering attributes are additive. HDA (contrarily to DA and AE) allows several browsers, each with its different filter. root Leaves Object Log 1 Object Log 2 Area 5 Log 3 Area 51 Area 52

HDA: General attributes of the items Code Attributes Description 1 DATA_TYPE data type for the item. See the definition of a VARIANT for valid values (VT_R4, etc.) 2 DESCRIPTION text that describes the item. 3 ENG_UNITS label to use in displays to define the units for the item (e.g., kg/s). 4 STEPPED whether data from the history repository should be displayed as interpolated (sloped lines between points) or stepped (vertically-connected horizontal lines between points) data. 5 ARCHIVING whether historian is recording data for this item (0 means no). 6 DERIVE_EQUATION equation to be used by a derived item to calculate its value. This is a free-form string. 7 NODE_NAME machine which is the source for the item. This is intended to be the broadest category for defining sources. For an OPC Data Access Server source, this is the node name or IP address of the server. For non-OPC sources, the meaning of this field is server-specific. 8 PROCESS_NAME process which is the source for the item. This is intended to the second-broadest category for defining sources. For an OPC DA server, this would be the registered server name. For non-OPC sources, the meaning of this field is server-specific. 9 SOURCE_NAME name of the item on the source. For an OPC DA server, this is the ItemID. For non-OPC sources, the meaning of this field is server-specific. 10 SOURCE_TYPE sort of source produces the data for the item. For an OPC DA server, this would be "OPC". For non-OPC sources, the meaning of this field is server-specific. 11 NORMAL_MAXIMUM upper limit for the normal value range for the item, is used for trend display default scaling and exception deviation limit calculations, should be the normal high value for the item. 12 NORMAL_MINIMUM lower limit for the normal value range for the item, is used for trend display default scaling and exception deviation limit calculations, should be the normal low value for the item. 13 ITEMID item identifier. This is used to allow filtering in the CreateBrowse method.

HDA: Attributes that affect how data are archived Code Attributes Data Type Description 14 Max Time Interval VT_FILETIME Specifies the maximum interval between data points in the history repository regardless of their value change.A new value shall be stored in history whenever OPCHDA_MAX_TIME_INT seconds have passed since the last value stored for the item. 15 Min Time Interval VT_FILETIME Specifies the minimum interval between data points in the history repository regardless of their value change.A new value shall be not be stored in history unless OPCHDA_MIN_TIME_INT seconds have passed since the last value stored for the item. 16 Exception Deviation VT_R8 Specifies the minimum amount that the data for the item must change in order for the change to be reported to the history database. See OPCHDA_EXCEPTION_DEV_TYPE for the specific meaning of this field. 17 Exception Deviation Type VT_I2 Specifies whether the OPCHDA_EXCEPTION_DEV is given as an absolute value, percent of span, or percent of value. The span is defined as OPCHDA_HIGH_ENTRY_LIMIT - OPCHDA_LOW_ENTRY_LIMIT. 18 High Entry Limit VT_R8 Specifies the highest valid value for the item. A value for the item that is above OPCHDA_HIGH_ENTRY_LIMIT cannot be entered into history. This is the top of the span. 19 Low Entry Limit VT_R8 Specifies the lowest valid value for the item. A value for the item that is below OPCHDA_LOW_ENTRY_LIMIT cannot be entered into history. This is the zero for the span.

HDA: Selecting the items Each server has only one group of items, the collection OPCHDAItems. By adding an OPCHDAItemID to that collection, the client defines its client handle and receives the server's server handle. The OPCHDAItemID (a string) should be retrieved by using the browser (like in DA or AE), since entering a string directly is unsafe. The server handle is used to access the log. Read operations are applied to the whole collection OPCHDAItems. .... Set colOPCHDAItems = MyOPCHDAServer.OPCHDAItems colOPCHDAItems.AddItem OPCHDAItemID ClientHandle colOPCHDAItems.SyncReadRaw .....

HDA: Reading the data When reading the data, the client has the choice to read synchronously or asynchronously (if the server permits the latter). The client specifies the time interval (Start time and end time) and whether the end values of that interval shall be included. The client has the choice to read the raw data or to apply an aggregation to the read data. The aggregations (see next slides) allow to reduce data.

HDA: Aggregates Symbol Description 0 NoAggregate Do not retrieve an aggregate (use only start time - stop time and filtering). 1 Interpolative Get interpolated values. 2 Total Get the totalized value (time integral) of the data over the resample interval. 3 Average Get the average data over the resample interval. 4 TimeAverage Get the time weighted average data over the resample interval. 5 Count Get the number of raw values over the resample interval. 6 StDev Get the standard deviation over the resample interval. 7 MinimumActualTime Get the minimum value in the resample interval and the timestamp of the minimum value. 8 Minimum Get the minimum value in the resample interval. 9 MaximumActualTime Get the maximum value in the resample interval and the timestamp of the maximum value. 10 Maximum Get the maximum value in the resample interval. 11 Start Get the value at the beginning of the resample interval (time stamp). 12 End Get the value at the end of the resample interval (time stamp of end of the interval). 13 Delta Get the difference between the first and last value in the resample interval. 14 RegSlope Get the slope of the regression line over the resample interval. 15 RegConst Get the intercept of the regression line over the resample interval, (time start of interval). 16 RegDev Get the standard deviation of the regression line over the resample interval. 17 Variance Get the variance over the sample interval . 18 Range Get the difference between the minimum and maximum value over the sample interval. 19 DurationGood Get the duration (in seconds) of time in the interval during which the data is good. 20 DurationBad Get the duration (in seconds) of time in the interval during which the data is bad. 21 PercentGood Get the percent of data (1 = 100 %) in the interval which has good quality. 22 PercentBad Get the percent of data (1 = 100 %) in the interval which has bad quality. 23 WorstQuality Get the worst quality of data in the interval. 24 Annotations Get the number of annotations in the interval.

This is useful to display smoothly trend diagrams. HDA interpolation interpolation is an aggregate used when the time scale of the derived log is smaller than that of the parent log. = parent log resample interval of the new log = filtered log time This is useful to display smoothly trend diagrams.

HDA corrections and annotations These features are seldom implemented. Basically, the values are accessed using the server handles that OPCHDAItems.AddItem returned.

HDA Automation Model OPC Common Overview: usage and specifications OPC as an integration tool Clients and Servers: configuration OPC Technology, client and custom interface OPC Data Access Overview: browsing the server Objects, types and properties Communication model Simple Programming Example Standard and components OPC Alarms and Events Specification Overview: definitions and objects Events Alarm Conditions Automation Model OPC Historical Data Specification Overview

HDA: Automation Interface summary OPCHDAServer An instance of an OPC HDA Server. An OPCHDAServer object must be created to can get references to other objects. A collection containing all the OPCHDAItem objects this client has created within the scope of the OPCHDAServer object OPCHDAItems (col) An object that maintains the item’s definition. Note the Custom Interface does not provide a separate Item Object. OPCHDAItem OPCHDAItem OPCHDABrowsers An object that browses item names in the server’s configuration. An collection containing a series of either OPCHDAValue or OPCHDAEntry objects, representing historical values of an item. OPCHDAHistory OPCHDAValue An object that represents a discrete historical value for an item or an attribute. OPCHDAValue OPCHDAEntry An object that extends the OPCHDAValue object to include additional information OPCHDAEntry

HDA Browsing and filtering The filter criteria are based on attributes (see the preceding slides). Attributes cannot be modified afterwards. Set MyOPCHDABrowser = MyOPCHDAServer.Createbrowser (NumCriteria, AttributeIDs, operatorCode, Filters, Errors MyOPCHDABrowser.MoveDown (MyOPCHDABrowser.OPCHDABranches(1)) Items = MyOPCHDABrowser.OPCHDAItems .... list these items

HDA: Object Interfaces (1)

HDA: Object Interfaces (2)

To probe further…. OPC Foundation: Specifications http://www.opcfoundation.org SoftwareToolbox Examples in Visual Basic http://www.softwaretoolbox.com/Tech_Support/TechExpertiseCenter/OPC/opc.html The Code Project OPC and .NET http://www.codeproject.com/useritems/opcdotnet.asp Matrikon Free client and server: http://www.matrikon.com WinTech Toolkit for an OPC server http://www.win-tech.com/html/opcstk.htm NewAge Automation Toolkit for an OPC server http://www.newageautomation.com

To probe further…. OPC Foundation: Specifications http://www.opcfoundation.org SoftwareToolbox Examples in Visual Basic http://www.softwaretoolbox.com/Tech_Support/TechExpertiseCenter/OPC/opc.html The Code Project OPC and .NET http://www.codeproject.com/useritems/opcdotnet.asp Matrikon Free client and server: http://www.matrikon.com WinTech Toolkit for an OPC server http://www.win-tech.com/html/opcstk.htm NewAge Automation Toolkit for an OPC server http://www.newageautomation.com