1. Software and Systems Division IHE-PCD Cycle 4 Test Overview, Strategy, and Tutorial NIST HL7 V2 Test Tools IHE-PCD Cycle 4 Test Overview, Strategy, and Tutorial WebEx (11 am – 12 pm, EST) 19 October 2009

2. Software and Systems Division NIST IHE-PCD V2 Test Effort NIST IHE-PCD V2 Test Effort NIST Team Members John Garguilo (john.garguilo@nist.gov,john.garguilo@nist.gov 301-975-5248) Sandra Martinez (sandra.martinez@nist.gov,sandra.martinez@nist.gov 301-975-3579) Maria Cherkaoui (maria.cherkaoui@nist.govmaria.cherkaoui@nist.gov Guest Researcher) NIST HL7 V2 Tool Web site: To be announced shortly…

3. Software and Systems Division What this ‘Tutorial’ is… Intended to promote a test strategy to meet key testing objectives (for IHE-PCD domain) Communicate a recommended common path forward to IHE- PCD participants, Domain Test Manager, and Test Tool developers –More comprehensive documentation –Feed-back mechanism into more complete technical framework and standards A focused testing approach for IHE-PCD –Cycle 4 “Pre-connectathon” –HL7 V2 (2.6) messages –Using a “centralized testing approach”, no firewall issues –To show conformance of vendor produced messages to specifications HL7 Standard IHE-PCD TF and Supplements hRTM IHE-PCD Pre-connectathon Test Cases Designed for IHE-PCD Vendor ease-of-use

4. Software and Systems Division What this ‘Tutorial’ is NOT… Focused on other IHE domains (i.e., outside of IHE-PCD) Training on a “polished” tool A lesson in HL7 A replacement for Message Workbench (MWB) –Everything you can do with the NIST HL7 V2 IHE-PCD tool, can be done in MWB A validation approach for ‘consumer’ type actors An HL7 Profile building tool –…at least not yet!

5. Software and Systems Division IHE-PCD Testing IHE-PCD Testing – Key Objectives Increase test comprehensiveness & quality Support both conformance & interoperability testing Support for pre- & virtual- connectathons, actual connectathon & enable year round testing Remain in alignment with IHE-PCD integration profile development road map Establish single framework for PCD covering increasing complexity and technologies over next 5 years Coordinate with IHE “Gazelle Project” Generate work products that companies can use in their regulatory submissions

6. Software and Systems Division IHE-PCD Testing IHE-PCD Testing – Key Ingredients Well Defined Integration Profiles –Technical Framework –Supplements Unambiguous Standards Test Guidelines Test Plans –Test Scenarios, Actors, Transactions, Validation Criteria Test Artifacts, including: –HL7 Profile(s) (and eventually x73 Device Specializations?) –Repositories –Nomenclature (e.g., RTM) –Value Tables (e.g., HL7, units, local, etc.), Default and Sample Values Test Cases

7. Software and Systems Division NIST Testing Strategy Overview NIST’s IHE-PCD Test Strategy –NIST Framework for Testing Distributed Healthcare Applications –Documentation: Test Guideline and Test Plans Provides test assertions for tooling –IHE-PCD “NIST HL7 V2 Testing Framework” Instance Coming Soon…Web Site: Coming Soon… –Cycle 4 Pre-connectathon (and virtual-connectathon) Potential use at IHE ’10 (January) Connectathon –Supports Conformance & leads toward Interoperability

8. Software and Systems Division NIST Testing Strategy Test Environments Instance Testing –Conformance (e.g., against HL7 2.x or CDA) Implementation conforms to Spec. on which it is based IHE Model: ~Virtual and Pre-Connectathon Isolated System Testing –Includes Instance Testing Activities –Protocol Conformance –Functional Behavior Conformance Features and Operational behavior correspond to Specs. IHE Model: ~Virtual and Pre-Connectathon Peer-to-Peer System Testing –Includes Isolated System Testing Activities –Interoperability Testing Testing complete application environment May include interacting w/ Database, using Network Communications, or interacting w/ other hardware, apps, or systems if appropriate IHE Model: ~Connectathon

9. Software and Systems Division Test Environment Message Validation NIST V2 Testing Tools: IHE-PCD Validation of IHE-PCD message(s) and corresponding HL7 Profile(s) Syntax and Semantic Content Validation –Against HL7 V2 message (e.g., PCD-01) Message structure (e.g., MSH,PID,PV1,OBR,NTE,{{OBX},OBX,OBX,OBX,…}) –Against HL7 profile (Msg_type^Event_type^ e.g., ORU^R01^…) –Against HL7 and/or user provided tables Example of user provided table is RTM for Ref_IDs, Units, etc. –Against ‘validation context’, including specific values Defined in XML (e.g., specific test case values)

10. Software and Systems Division NIST IHE-PCD Testing Strategy Overview Validation Context –Integrated via context validation files for this year’s cycle –Future, likely a web service (call to repository) Going forward – RTMMS (and tied to ISO/IEEE 11073)… Integrated as part of NIST Test Framework  Framework/Supplement (Transactions/HL7 Messages)  HL7 Profile (one for each Integration Profiles)  RTM Terminology (hRTM) –Integrated via context validation files for this year’s cycle Future, likely a web service (call to repository) Going forward – RTMMS (RTM Management System) –Tied to ISO/IEEE x73 (via ballot process)… –Called out in Technical Framework and Supplements Integrated as part of NIST Test Framework

11. Software and Systems Division Validation against ‘failure types’: –VERSION*: The version in the message and in the profile should match. –MESSAGE_STRUCTURE_ID*: The message type (MSH.9 element) in the profile and in the message should match. –MESSAGE_STRUCTURE: The message should have a valid message structure (correct usage, correct cardinality, and correct element name). –USAGE: R elements should be present; X elements should not be present in the message. –CARDINALITY: Elements should be present at least the minimum times and at most the maximum times specified in the profile. It should also take into account the usage of the element (X element with a minimum of 4 should not be present in the message). –LENGTH: The value of the element should have a length equal or less than the value specified in the profile. –DATATYPE: For the datatype NM, DT, DTM, SI and TM, the value of the element should match the regular expression defined in the standard. –DATA: The value of the element should match a constant specified in the profile, a value set specified in a table, a value or a regular expression specified in the message validation context. –MESSAGE_VALIDATION_CONTEXT*: This is a user input error when the location specified in the message validation context can't be found in the message. –TABLE_NOT_FOUND*: This is a user input when a table can't be found in the table files (TableProfileDocument). –AMBIGUOUS_PROFILE*: The profile should not be ambiguous. NIST V2 Testing Tools and Services Testing Validation Types

12. Software and Systems Division NIST IHE-PCD Test Guidelines (’09-’10 Pre-connectathon) Quick, executive summary level view Defines testing requirements for those performing testing –HL7 Messaging version (e.g., V2.6) –HL7 profile Identifies test environment –See ‘Test Environment’ slides above… Identifies what types of testing shall occur Links test artifact(s) (needed) to validate specific test case(s) Syntax Validation –HL7 Syntax/structure – basic syntax via HL7 V2.6 standard –HL7 IHE-PCD Profile Validation – required fields, order, data types, cardinality via message profile Semantic Validation(via context validation file) –Integration Profile Validation – (semantic) via message profile file (XML) –Rosetta Validation – (semantic) terminology w/ co-constraints from RTM Integration Profile –Test case validation – (semantic) specific field values

13. Software and Systems Division NIST IHE-PCD Test Plans (’09-’10 Pre- and Virtual- Connectathon) Clear and concise documentation Mirror (point to) and/or include test cases –Already defined and new test cases as developed/introduced Necessary for rigorous HL7 profile and message validation –Defines “validation context” specifications used by tooling Recommended document template as we go forward… –Used by IHE-PCD Integration Profile working groups to establish exactly what’s to be tested Communicates specific validation criteria to participating vendors –I.e., no surprises …Supporting documentation for work products –…that perhaps companies can use in their regulatory submissions Start w/ static messages and move into scenarios –Way to identify appropriate Rosetta Terminology according to IHE use case …Perhaps be adopted by other domains as “conformance” documentation model…

14. Software and Systems Division NIST IHE-PCD Test Plans Issue Resolution Such documentation will lead to issues –Identify gaps and ambiguities –Identify road-map work items/tasks Feedback from Test Plan and HL7 profile work –Likely lead to CPs to Technical Framework and Supplements Further development (perhaps via website) to: –Update Forms/Tables (via ‘approved’ IHE-PCD process) –Forms/Tables  > XML generation Basis for HL7 Profile over IHE cycles –Provide up-to-date electronic version

15. Software and Systems Division IHE-PCD Cycle 4 Testing Documentation Test Guideline Documents NIST DEC Pre-Connectathon Test Guidelines.docx NIST ACM Pre-Connectathon Test Guidelines.docx NIST IDCO Pre-Connectathon Test Guidelines.docx NIST PIV Pre-Connectathon Test Guidelines.docx Test Plan Documents DEC_TEST PLAN.docx DEC_SPD_TEST PLAN.docx ACM_TEST PLAN.docx IDCO_TEST PLAN.docx PIV_TEST PLAN.docx  Note: All documents are first cut/version and need additional review…

16. Software and Systems Division NIST-produced HL7 Profiles (’09-’10 Pre-connectathon) NIST HL7 Message Profiles (XML format) –To be used with NIST IHE-PCD HL7 V2 Validation Tooling… –Website: coming soon… –Currently vetting w/ Messaging Workbench (MWB) Naming Convention Used: –*NIST_[w...w]_xxx_yyy-NN_zzz^TTT.xml –Example 1: NIST_IDCO_DOR_PCD-09_ORU^R01.xml –Example 2: NIST_SPD_DOC_PCD-02_QSB^Q16.xml [w...w]: Integration Profile Group/Option (e.g., “PIV”) xxx: Actor (e.g., “IOP”) yyy-NN: IHE-PCD transaction-Number (e.g., “PCD-02”) zzz^TTT: Message Type^Trigger Event (e.gs., “ROU^R01”, “RGV^O15”) *MWB is primary tool this cycle used by vendors, so keep “NIST” label to avoid confusion.

17. Software and Systems Division NIST IHE-PCD HL7 Profiles (cycle 4) NIST_DEC_DOR_PCD-01_ORU^R01.xml NIST_SPD_DOC_PCD-02_QSB^Q16.xml NIST_PIV_IOP_PCD-03_RGV^O15.xml NIST_ACM_AR_PCD-04_ORU^R01.xml NIST_IDCO_IDCR_PCD-09_ORU^R01.xml

18. Software and Systems Division Profile Validation Feedback to TF/Supplements… NIST_DEC_DOR_PCD-01_ORU^R01.xml Remaining errors concern issues related to contradictions with HL7 v2.5 versus v2.6 –(captured using HL7Web application) Errors (leading to CPs in the Framework/Supplement docs…) –Error1: segment PV1 is ‘R’ Required by the standard but Optional in DEC –Error2: segment TQ1 is ‘R’ Required by the standard but Optional in DEC –Error3: segment SPM is ‘R’ Required by the standard but Not Supported in DEC –Error4: OBX-20 is not defined in the v2.5 standard (it is in v2.6) Documented Issues Please send issues to: john.garguilo@nist.gov

19. Software and Systems Division Message Validation Context Technical Framework/Supplement Validation Naming Convention: w…w-yyy-nn-ValidationContext.xml –[w...w]: Integration Profile Group/Option (e.g., “PIV”) –yyy-nn: IHE-PCD transaction-Number (e.g., “PCD-03”) DEC-PCD-01-ValidationContext.xml SPD-PCD-02-ValidationContext.xml PIV-PCD-03-ValidationContext.xml ACM-PCD-04-ValidationContext.xml IDCO-PCD-09-ValidationContext.xml

20. Software and Systems Division Message Validation Context hRTM Validation hRTM –Integrated via context validation files for this year’s cycle hDEC-PCD-01-ValidationContext.xml hPIV-PCD-03-ValidationContext.xml hACM-PCD-04-ValidationContext.xml Needed: IDCO-PCD-09ValidationContext.xml

21. Software and Systems Division NIST Test Case Message Validation Context (’09-’10 Pre-connectathon) NIST validation of specific values (derived from the test cases for each Integration Profile) –Based on Use Case –Instance of Use Case ultimately demonstrated at Showcase NIST V2 Tools currently only handle static message validation from reporter-type actors…

22. Software and Systems Division NIST Test Case Message Validation Context (’09-’10 Pre-connectathon) NIST validation of specific values (derived from the test cases for each Integration Profile) –Based on Use Case –Instance of Use Case ultimately demonstrated at Showcase DEC (DOR for PCD-01) 60001-ValidationContext.xml 60002-ValidationContext.xml 60003-ValidationContext.xml 60004-ValidationContext.xml 60005-ValidationContext.xml 60006-ValidationContext.xml 60007-ValidationContext.xml 60008-ValidationContext.xml 60009-ValidationContext.xml 60010-ValidationContext.xml 60041-ValidationContext.xml

23. Software and Systems Division NIST Test Case Message Validation Context (’09-’10 Pre-connectathon) NIST validation of specific values (derived from the test cases for each Integration Profile) –Based on Use Case –Instance of Use Case ultimately demonstrated at Showcase PIV (IOP for PCD-03) 60101-ValidationContext.xml 60102-ValidationContext.xml 60103-ValidationContext.xml 60104-ValidationContext.xml 60105-ValidationContext.xml 60106-ValidationContext.xml 60107-ValidationContext.xml 60108-ValidationContext.xml 60109-1-ValidationContext.xml 60109-2-ValidationContext.xml 60109-3-ValidationContext.xml 60109-4-ValidationContext.xml 60110-ValidationContext.xml

24. Software and Systems Division MSH|^~\&|PAT_DEVICE_BBRAUN^0012210000000001^EUI64|BBRAUN|OTHER_GE_CPA|GEHEALTHCARE|20 061215111017-0600||ORU^R01^ORU_R01|19|P|2.5|||NE|AL||ASCII|EN^English^ISO659||IHE PCD ORU- R01 2006^HL7^2.16.840.1.113883.9.n.m^HL7 PID|||CD60002^^^IHE^PI||Darwin^Charles^^^^^L|Emerine^^^^^^L|19620101|M OBR|1|AB12345^HL7^ACDE48234567ABCD^EUI-64|CD12345^HL7^ACDE48234567ABCD^EUI- 64|ServiceID^ServiceID|||20061215111014-0600 OBX|1||69985^MDC_DEV_PUMP_INFUS_MDS^MDC|1000001.0.0.0|||||||X|||||N60002||X^^A0001^BBRAUN OBX|2||69986^MDC_DEV_PUMP_INFUS_VMD^MDC|1000001.1.0.0|||||||X OBX|3||126978^MDC_DEV_PUMP_INFUS_CHAN_DELIVERY^MDC|1000001.1.1.0|||||||X OBX|4||126977^MDC_DEV_PUMP_INFUS_CHAN_SOURCE^MDC|1000001.1.2.0|||||||X OBX|5||126977^MDC_DEV_PUMP_INFUS_CHAN_SOURCE^MDC|1000001.1.3.0|||||||X OBX|6|ST|184504^MDC_PUMP_MODE^MDC|1000001.1.1.101|pump-mode- nominal||||||R|||20061215111014-0600|||||20061215111014-0600 OBX|7|ST|184508^MDC_PUMP_STAT^MDC|1000001.1.1.102|pump-status- infusing||||||R|||20061215111014-0600|||||20061215111014-0600 OBX|8|NM|157784^MDC_FLOW_FLUID_PUMP^MDC|1000001.1.1.103|100.0|265266^MDC_DIM_MILLI_L_PE R_HR^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|9|NM|157784^MDC_FLOW_FLUID_PUMP^MDC|1000001.1.3.301|100.0|265266^MDC_DIM_MILLI_L_PE R_HR^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|10|NM|157872^MDC_VOL_FLUID_TBI_REMAIN^MDC|1000001.1.3.302|9.8|mL^mL^UCUM^263762^M DC_DIM_MILLI_L^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|11|NM|157916^MDC_TIME_PD_REMAIN^MDC|1000001.1.3.303|5|min^min^UCUM^264352^MDC_DIM _MIN^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|12|ST|184330^MDC_DRUG_NAME_TYPE^MDC|1000001.1.3.304|Amoxicillin||||||R|||20061215111014- 0600|||||20061215111014-0600 Example HL7 Message (ER7 format)

25. Software and Systems Division MSH|^~\&|PAT_DEVICE_BBRAUN^0012210000000001^EUI- 64|BBRAUN|OTHER_GE_CPA|GEHEALTHCARE|20061215111017- 0600||ORU^R01^ORU_R01|19|P|2.5|||NE|AL||ASCII|EN^English^ISO659||IHE PCD ORU- R01 2006^HL7^2.16.840.1.113883.9.n.m^HL7 PID|||CD60002^^^IHE^PI||Darwin^Charles^^^^^L|Emerine| 19620101 |M OBR|1|AB12345^HL7^ACDE48234567ABCD^EUI-64|CD12345^HL7^ACDE48234567ABCD^EUI- 64|ServiceID^ServiceID|||20061215111014-0600 OBX|1||69985^MDC_DEV_PUMP_INFUS_MDS^MDC|1000001.0.0.0|||||||X|||||N60002|| ^^A0001^BBRAUN OBX|2||69986^MDC_DEV_PUMP_INFUS_VMD^MDC|1000001.1.0.0|||||||X OBX|3||126978^MDC_DEV_PUMP_INFUS_CHAN_DELIVERY^MDC|1000001.1.1.0|||||||X OBX|4||126977^MDC_DEV_PUMP_INFUS_CHAN_SOURCE^MDC|1000001.1.2.0|||||||X OBX|5||126977^MDC_DEV_PUMP_INFUS_CHAN_SOURCE^MDC|1000001.1.3.0|||||||X OBX|6|ST|184504^MDC_PUMP_MODE^MDC|1000001.1.1.101|pump-mode- nominal||||||R|||20061215111014-0600|||||20061215111014-0600 OBX|7|ST|184508^MDC_PUMP_STAT^MDC|1000001.1.1.102|pump-status- infusing||||||R|||20061215111014-0600|||||20061215111014-0600 OBX|8|NM|157784^MDC_FLOW_FLUID_PUMP^MDC|1000001.1.1.103|100.0| 3122 ^mL/h^UCUM^265266 ^MDC_DIM_MILLI_L_PER_HR^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|9|NM|157784^MDC_FLOW_FLUID_PUMP^MDC|1000001.1.3.301|100.0| 3122 ^mL/h^UCUM^265266 ^MDC_DIM_MILLI_L_PER_HR^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|10|NM|157872^MDC_VOL_FLUID_TBI_REMAIN^MDC|1000001.1.3.302|9.8| 1618 ^mL^UCUM^26376 2^MDC_DIM_MILLI_L^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|11|NM|157916^MDC_TIME_PD_REMAIN^MDC|1000001.1.3.303|5| 2208 ^min^UCUM^264352^MDC_ DIM_MIN^MDC|||||R|||20061215111014-0600|||||20061215111014-0600 OBX|12|ST|184330^MDC_DRUG_NAME_TYPE^MDC|1000001.1.3.304|Amoxicillin||||||R|||20061215111014- 0600|||||20061215111014-0600 Example HL7 Message (ER7 format-infused errors)

26. Software and Systems Division NIST V2 Tools (via Web) Demonstration NIST HL7 Web (for IHE-PCD)NIST HL7 Web (for IHE-PCD) (website URL coming soon…) Select the IHE Profile Select Sending Actor Transaction Pre-Connectathon Test Case Browse for a message or cut and paste message into text box Select ‘Load Message’ Select ‘Validation’ under “Start Validation” heading Generate and review resulting ‘Report’ –Review errors

27. Software and Systems Division Issues to resolve… For consideration as we move forward… V2.5 move to V2.6 Tool Usage Feedback mechanism –Send email to NIST (see last slide for contacts) CPs to IHE-PCD Technical Framework/Supplements –Learned from profile exercise …

28. Software and Systems Division Next Steps Review Test Guide and Test Plan documents “Test” the test tools (through usage by IHE- PCD members) –Start with a few volunteers Complete Test Cases and add to interface Provide NIST V2 Tool web site URL Develop NIST HL7 V2 IHE-PCD ‘User Guide’ Determine Feedback mechanism for IHE- PCD members Continue to update profile and add test case validation (as IHE-PCD updates TF/Supplements)

29. Software and Systems Division NIST IHE-PCD V2 Test Effort NIST IHE-PCD V2 Test Effort NIST Team Members John Garguilo (john.garguilo@nist.gov,john.garguilo@nist.gov 301-975-5248) Sandra Martinez (sandra.martinez@nist.gov,sandra.martinez@nist.gov 301-975-3579) Maria Cherkaoui (maria.cherkaoui@nist.govmaria.cherkaoui@nist.gov Guest Researcher) NIST HL7 V2 Tool Web site: To be announced shortly…

30. Software and Systems Division Thank YOU! Recommend Test Plan approach to define validation criteria across Integration Profiles Recommend HL7 Profile approach –To be used as baseline in each cycle going forward… –Easy for new companies to get up and running… –IHE-PCD profiles already integrated into NIST V2 tooling as we go forward… Questions? Note: ITI PIX/PDQTool tutorial scheduled… Thanks for your attention

31. Software and Systems Division Extra Slides

32. Software and Systems Division Test Artifacts Conformance Profiles HL7 Tables Validation Context Files Generation Context Files HL7 V2 Message Validation Services Test Management Test Harness (Java Code) HL7 V2 Message Validation Criteria Results HL7 V2 Message Validation Report User Instance Testing HL7 V2 Message Validation Test Execution Web Application Client Report HL7 V2 Message Conformance Testing: Showing that messages are in compliance w/ HL7 V2.x and constraints specified in IHE-PCD framework documents.

33. Software and Systems Division Test Artifacts Conformance Profiles HL7 Tables Validation Context Files Generation Context Files Services Test Management Test Harness (Java Code) HL7 V2 Message RTM Validation Criteria Results RTM Message Validation Report User Instance System Testing IHE-PCD RTM Validation Test Execution Web Application Client Report HL7 V2 Message HL7 V2 Message Validation RTM Message Validation IHE PCD DOC Test Agent Conformance Testing: Showing that messages are in compliance w/ Harmonized-Rosetta terminology SecurityTime

34. Software and Systems Division IHE-PCD DOR System Future: Isolated System Testing IHE-PCD Application Functional Behavior Testing Test Artifacts Conformance Profiles HL7 Tables Validation Context Files Generation Context Files IHE-PCD DOC Test Agent HL7 V2 Message Generation IHE-PCD DOR Test Agent HL7 V2 Message Validation Services Test Management Test Harness (Java Code) Router IHE-PCD DEC DOC Test Scenario Results Message Validation Reports Vendor Test Execution Web Application Client Report RTM Message Validation Conformance Testing: May include HL7 message syntax validation, RTM, and/or application functional behavior testing SecurityTime

35. Software and Systems Division Test Artifacts Conformance Profiles HL7 Tables Validation Context Files Generation Context Files Services Test Management Test Harness (Java Code) HL7 V2 Message RTM Validation Criteria Results RTM Message Validation Report User Future: Isolated System Testing IHE-PCD RTM Validation Test Execution Web Application Client Report HL7 V2 Message HL7 V2 Message Validation RTM Message Validation IHE PCD DOC Test Agent Conformance Testing: Showing that messages are in compliance w/ Harmonized-Rosetta terminology SecurityTime IHE-PCD DOR System HL7 V2 Message

36. Software and Systems Division Future: Peer-to-Peer System Testing IHE-PCD Application Functional Behavior Testing Test Artifacts Conformance Profiles HL7 Tables Validation Context Files Generation Context Files IHE-PCD DOC Test Agent HL7 V2 Message Generation IHE-PCD DOR Test Agent HL7 V2 Message Validation Services Test Management Test Harness (Java Code) Router/Logger/Proxy IHE-PCD DEC DOC Test Scenario Results Message Validation Reports Vendor Test Execution Web Application Client Report RTM Message Validation Conformance Testing: May include HL7 message syntax validation, RTM, and/or application functional behavior testing Interoperability Testing : system(s) (functional) behavior SecurityTime IHE-PCD DOR System IHE-PCD DOC System