1. HL7 V2 Conformance Testing Robert Snelick NIST January 20 th, 2004 rsnelick@nist.gov

2. NIST National Institute of Standards and Technology Department of Commerce Software Diagnostics and Conformance Testing Assist industry and government agencies in IT standard related activities development of standards reference implementations conformance testing tools Public Domain Explore what role we can play in HL7 conformance testing

3. Session Goals Explore HL7 V2 conformance testing issues Proposal for conformance testing based on the concept of message profiles Identify the gapswhat needs to be done? Feedback: What does conformance testing mean, exactly? What are the most important aspects to test? How much of an impact will it have on the community? How is testing done now? Should better conformance statements be created? What tools can be leveraged (MWB, HAPI, etc.)? What have others done (MWB, AHML, etc.)? Move forward? Refine testing plan

4. Overview Generate test message suites and test harness for conducting conformance tests Suite of Test Messages based on the concept of message profiles conformant message structures derived from profiles numerous message instances for each profile test cases and scenarios combine fixture-data and automation huge state space limited selective tests Conformance Testing testing framework based on test messages acts as a model or simulated system tool that generates and validates messages mechanisms to use messages batch file, tool, web-service Not Certification!

5. Test Messages Test messages to ensure that a system is conformance to a given profile Coherent design to systematically test implementation Covers the definition of the message profile Static definition of profile Dynamic definition of profile Reference Database of fixture data Used for message generation Public domain Test Cases and Test Scenarios Valid and (Invalid ?)

6. Test Cases Static Definition of Profile Message Level Profile segment definition (message structure) segment cardinality; Example: [0..1] segment optional, but can have one occurrence generate 3 messages with 0, 1, and 2 segments missing or extra data fields Segment Level Profile field length, usage (R, RE, C, CE, X), repetitions Field Level Profile coded entry composite data (components, sub-components) Content out-of-range data, table data, missing data boundary conditions, data type, more… Dynamic Definition of Profile Acknowledgements What else?

7. Test Scenarios Test behavior as specified by the standard Example: update, field X = A null field X = empty, i.e., deleted not-present field X = A String sequence of test messages together to test certain functional aspects of an implementation Example: swap patients, merge patient information Many others…

8. Path to Conformance Testing HL7 Messaging Standard Profile Specific Library e.g., MWB e.g., HAPI Source Generator Message Profile (XML Document) Test Message Testing Support (test cases, fixture data, data generator) Test Framework Message Factory

9. Test Report Profile (XML ) Generate Message Record Message Send Message Process ACK Message Send Query Request Message Parse Message Process Message Send ACK Message Process Query Send Query Response Report Result Analyze Process Response System Under TestTest Driver Conformance Testing Model A Conceptual Process compares actual vs. expected

10. HL7 Test Messages Reference Fixture Data Generator Test Messages Existing Components Proposed Components Profile (XML) Specific Java Class Library Message Factory Implementation Testing Framework Test Case Specifications Message Factory HAPI Source Generator

11. HL7 Test Messages Reference Data Generator Test Messages Reference Test System (Tool) Test Report System Under Test Existing Components Proposed Components Registered Profiles (XML) Specific Java Class Library Profile, Request Messages Deliver Messages* Send Messages for Validation * SUT determines use of messages Test System Design Test Cases Test Scenarios Message Factory HAPI Source Generator Send Query Response Query

12. Conformant HL7 Messages Reference Data Generator Test Messages Test Validator Test Service OR Test Report System Under Test Report Existing Components Proposed Components Need API Profiles (XML) Specific Java Class Library Separate Connection Round-trip Messages Test Service Design Test Cases HAPI Source Generator Message Factory Test Scenarios

13. Discussion Is there utility in creating the profile specific test messages (reference data, test cases, testing framework)? Are message profiles used in practice to make conformance testing worthwhile? In the Future? What are the most important aspects to test? What is the shelf-life for this service/tool? Can we achieve better conformance statements? How much can be automated? At what level? Is it feasible to automatically generate sensible test messages? Does this infringe upon someones business model?