1 NIST HL7 Conformance Testing Framework Justin Manweiler The College of William and Mary, class of 2007 Computer Science SURF Student Colloquium NIST – Gaithersburg, MD August 8-10, 2006 OU: ITL Division: SDCT (897) Advisor: Rob Snelick Promoting HL7 Compliance to Improve Care

2 Focus Current state of healthcare IT HL7 messaging standard The role of NIST Personal work

3 Motivation Safety: much room for improvement …  Up to 98,000 American deaths/year due to preventable errors (according to the Institute of Medicine)  Mere existence of healthcare IT does not solve problem Lack of communication between diverse systems Need for enhancement of information and data systems Goal Strong informatics standards (provided by the HL7 standard) Correct implementations (facilitated by NIST) Realization of benefits possible with healthcare IT (improved care)

4 Health Level Seven (HL7) Standard Enables key healthcare data exchange  Clinical and administrative messaging standard  Example message types: lab order, report request, patient transfer Widespread basic usage  90% US hospitals, international use growing Expanded compliance would lead to…  Superior quality of care  Improved patient safety  Reduction of costs

5 Major HL7 Shortcoming: Complexity Universal design problematic  Needed to gain broad support  Attempts to accommodate most business cases Overwhelmingly large  Many optional features  Systems difficult to implement/debug  Plug-and-play installations impossible  Described as “total chaos” Complexity limits effectiveness

6 Response: Message Profiles Allow for the customization/simplification of HL7  For a particular implementation  Limit scope to a manageable size  Specify optional constructs  Path to conformance testing System interoperability requires strict profile compliance by each endpoint Conformance testing critical

7 Message Profile... MSH EVN PID NK1 PV1 PV2 OBX AL1 ADT^A Fields/Components: -Field Usage (Optionality) (R, RE, C, CE, X) -Cardinality (max repeats) -Value Sets/Coding system -Descriptions... MSH EVN PID NK1 PV1 PV2 OBX AL1 Segments/Segment Groups: -Cardinality (min, max) Message Profile HL7 Message Structure PPR PPT PPV PRM PRR PTR QBP QCK QCN QRY QSB QSX QVR RAR RAS RCI RCL RDE RDR RDS RDY REF RER RGV ROR RPA RPI RPL RPR RQA RQC RQI RQP RQQ RRA RRD RRE RRG RRI RSP SIU SPQ SQM SRM SSR SSU SUR TBR TCR TCU UDM VQQ VXQ VXR VXU VXX ACK ADR ADT BAR CRM CSU DFT DOC DSR EAC EAN EAR EDR EQQ ERP ESR ESU INR INU LSR LSU MCF MDM MFD MFK MFN MFQ MFR NMD NMQ NMR OMD OMG OML OMN OMP OMS ORD ORF ORG ORL ORM ORN ORP ORR ORS ORU OSQ OSR OUL PEX PGL PIN PMU PPG PPP Message Types A01 A02 A03 A04 A05 A06 A07 A08 A09 A10 A11 A12 A13 A14 A15 A16 A17 A18 A19 A20 A21 A22 A23 A24 A25 A26 A27 A28 A29 A30 A31 A32 A33 A34 A35 A36 A37 A38 A39 A40 A41 A42 A43 A44 A45 A46 A47 A48 A49 A50 A51 Message Events Profile Complexity Testing must be profile specific Current practice: manual, expensive  Test suites generated by hand  Plug-and-play interoperability is often cost- prohibitive The Need for Dynamic Test Creation

8 The Role of NIST: Message Maker Created under Lead Scientist, Rob Snelick (Division 897) Dynamic test generator Produces a set of test messages  Based on a system’s specific profile  Form a customized testing suite <HL7v2xConformanceProfile H <MetaData Name="CALINX" Or ER7 <DynamicDef AccAck="NE" Ap <HL7MsgType=“ADT" EventType=“A01 <Segment Name="MSH" LongN <Field Name="Field Separator" Us <Field Name="Encoding Characters" </Reference <Field Name="Sending Application" <Component Name="namespace ID" Usage="X Message Maker MSH|^~\&|REGA EVN|A05| PID|1||191919^ NK1|1|MASSIE^E NK1|2|MASSIE^I … Test Messages Profile HL7 Standard HL7 Standard Real System

9 Personal Work Validation Infrastructure Profile Validation Support Message Validation Support  Basic direct message-to-standard validation  Message-to-standard validation chain Message-to-profile validation Profile-to-standard validation Supporting the Message Maker project.

10 Employed Technologies XML: eXtensible Markup Language  Similar in structure to HTML  Way of storing and describing data Java: object-oriented programming language  DOM: XML representation as tree data structure  XMLBeans: Java-to-XML data type binding XSLT: transformation of XML documents XML Schema: valid document structure definition Many XML technologies are utilized for validation as HL7 messages can be expressed as XML. Profiles are always in XML. HL7 XML Encoded Messages MSH|^~\&|REGA EVN|A05| PID|1||191919^ NK1|1|MASSIE^E NK1|2|MASSIE^I … HL7 ER7 Encoded Messages Conversion XML representation can be easily manipulated.

11 Profile Validation Strategy <HL7v2xConformanceProfile H <MetaData Name="CALINX" Or ER7 <DynamicDef AccAck="NE" Ap <HL7MsgType=“ADT" EventType=“A01 <Segment Name="MSH" LongN <Field Name="Field Separator" Us <Field Name="Encoding Characters" </Reference <Field Name="Sending Application" <Component Name="namespace ID" Usage="X XML HL7 Profile Validation Report XML Schema Validator Version-specific HL7 Profile XML Schema Profile-to-standard validation Schema Violation Interpreter Generated XML Schema HL7-provided Message XML Schemas (HL7-version-specific, message-structure-specific) Construction using DOM

12 Message Validation Strategy HL7 XML Encoded Messages Validation Messages Validation Report XML Schema Validator HL7-provided Message XML Schema Basic direct message-to-standard validation MSH|^~\&|REGA EVN|A05| PID|1||191919^ NK1|1|MASSIE^E NK1|2|MASSIE^I … HL7 ER7 Encoded Messages Conversion Refinement by DOM Modification Schema Violation Interpreter When a profile does not exist to validate against, HL7-provided schemas are used to perform basic validation.

13 Message Validation Strategy <HL7v2xConformanceProfile H <MetaData Name="CALINX" Or ER7 <DynamicDef AccAck="NE" Ap <HL7MsgType=“ADT" EventType=“A01 <Segment Name="MSH" LongN <Field Name="Field Separator" Us <Field Name="Encoding Characters" </Reference <Field Name="Sending Application" <Component Name="namespace ID" Usage="X XML HL7 Profile XSLT Transformation HL7 XML Encoded Messages Validation Messages Refinement by DOM Modification Validation Profile Validation Report XML Schema Validator Profile-specific Message XML Schema Message-to-profile validation MSH|^~\&|REGA EVN|A05| PID|1||191919^ NK1|1|MASSIE^E NK1|2|MASSIE^I … HL7 ER7 Encoded Messages Conversion Refinement by DOM Modification Schema Violation Interpreter Profile XLST Transformer

14 … /HL7v2xConformanceProfile/HL7v2xStaticDef/ (/*:ADT_A01/*:MSH)[1]/*:MSH.10 ((/*:ADT_A01/*:MSH)[1]/*:MSH.10)[2] Invalid content.... Cardinality error detected at Field 'Message Control ID'. Element must appear at least 1 time and no greater than 1 time in the message instance. … Message Validation Reporting Example error section of an XML message validation report. Reports provides as much detail as can be determined about detected errors. Location of the error Name of the error (classification) Textual description

15 Chaos Agreement Testability Verification Interoperability HL7 Standard HL7 Standard Goal: test suite Use Message Maker Build your own ($$$) Message Verification Benefits of the Message Maker Testing Framework Personal Work Impact: Message Profile Manual Test Suite  tests needed for each profile  written individually  meticulous work  high cost  often tests not performed Message Maker Test Suite  tests needed for each profile  automatically generated  easy  lower cost  increases likelihood tests will be performed Profile Verification

16 Acknowledgements My group, especially,  My advisor, Robert Snelick  Sydney Henrard The SURF ITL advisors,  Tim Boland  Larry Reeker  Ivelisse Aviles