PQDIF PQDIF: A Technical Overview Prepared by: Erich Gunther, Bill Dabbs, and Rob Scott Electrotek Concepts, Inc. NEW! IMPROVED!

PQDIF What are the goals of PQDIF? Provide a platform-neutral way to interchange Power Quality data between software packages and instruments. Be standardized by IEEE to provide a stable target for third-party developers.

PQDIF What other PQ formats are out there? IEEE COMTRADE –Excellent for waveform and simple variable trend data, but is not suitable for other types of PQ data such as min/max/avg RMS trends, harmonic spectra, probabilities, event logs, or lightning flash data. PQView ® and other databases –Excellent for analysis and storage of massive quantities of data, but not for an interchange format.

PQDIF The Desired Characteristics Simple Extensible … but without breaking old apps Compact … preferably binary … optional compression Flexible

PQDIF The Basic Design A PQDIF file is made up of a set of “records” which are logically related: –Data Source (generally an instrument) –Monitor Settings (optional) –Observations Within each record, there is a set of “elements” which define the contents of the record.

PQDIF The Record Structure The “high-level” record structure is completely separated from the “low-level” element structure. –The overall structure of the file can be parsed quickly without having to read the entire file. This will also allow us to use different high- level structures in the future (storing records directly in a database, for example).

PQDIF The Element Structure The internal record structure is made up of a set of tagged data “elements.” Because each element is tagged, there is a separation between the “physical” and “logical” structure. This allows the logical structure to be defined separately from the physical structure.

PQDIF The four aspects of the design

PQDIF High-level logical format The records are strung together in an implicit hierarchy, based on the record type tags. –The file starts with a Container record, which “owns” any number of Data Source records. –Each Data Source record owns Monitor Settings and Observation records.

PQDIF The high-level physical format At the highest level, the file is made up a series of linked records. This linked nature allows manipulations to the record structure without having to rewrite the entire file.

PQDIF The high-level physical format: Inserting a record We can insert a new record -- even if it is physically at the end of the file.

PQDIF The high-level physical format: Deleting records We can also delete a record by adjusting the links to skip around it.

PQDIF The high-level physical format: Reordering records Likewise, we can reorder existing records by adjusting the links.

PQDIF Internal record structure Each record is made up of a “header” and a “body.” –The header provides the high-level information. –The body contains the low-level information.

PQDIF Low-level physical format The body of each record is made up of a set of “elements” which contain data. There are three elements: –Scalar represents a single data value –Vector represents an array of data values –Collection contains other elements

PQDIF The low-level physical format: Identifying elements In order to be able to identify a specific element, each one is given a “tag.” This tag is actually a GUID*, which makes extending the list of tags virtually foolproof. *A “Globally Unique Identifier” is a 16-byte integer which is defined by a standard algorithm. Any computer in the world can generate a GUID and be reasonably assured that it is absolutely unique.

PQDIF The low-level physical format: Think of it as a file system... A tag is a file name. A collection is a subdirectory. A scalar is a file with one number in it. A vector is a file with lots of numbers (or a string) in it. A subdirectory (collection) can contain another subdirectory. Contents: Contents: Count: 2 0: 110 1: 235 Contents: “PQNode 8010”

PQDIF The low-level physical format: Primitive data types –Signed integer (1, 2 or 4 bytes) –Unsigned integer (1, 2 or 4 bytes) –Boolean (1, 2 or 4 bytes) –Real (4-byte single or 8- byte double precision) –Complex (8-byte single or 16-byte double precision) –Character (1-byte ASCII or 2-byte Unicode) –Date stamp (12 bytes) –GUID (16 bytes) A scalar or vector element can hold values of the following types:

PQDIF Low-level logical format The low-level physical format allows each element to have a tag. The logical format defines what the tags are, and what element type and primitive data type(s) are expected for each. Since collections can contain other collections, the logical format must also specify this hierarchy.

PQDIF An example...

PQDIF Benefits of a tagged format The tagged element structure provides a great degree of flexibility and allows the contents to be defined logically independently of the actual physical definition.

PQDIF Work Completed Physical format finalized. Version 1.5 logical format finalized and documented Third generation COM library for manipulating PQDIF 1.5 complete (written in C++) Simple PQDIF reading and writing code examples suitable for embedded systems complete (written in C) Command line tool to dump PQDIF structure and data. Program to convert Square D DaDisp based waveform files and PASS 8010 Transient files to PQDIF. Examples of PQDIF viewers (one VB example and one Microsoft MFC example with source).

PQDIF Work still to be done More tools and examples. –COMTRADE converter –PASS 8010 PQNode Converter - support more types. More field trials. –Salt River Project - PML translator –TVA - substation monitoring project –Carolina Power and Light Provide leadership, documentation and sample code and for IEEE standards process.

PQDIF Conclusion PQDIF meets all of our goals. If handled properly, PQDIF can become a widely-accepted, stable format. With appropriate software tools, it appears to solve a number of technical issues for our projects (PQDS, for example).