1. A Supplier View of Testing Integration with National Systems Conan Ask Jean-Philippe Lemoussu

2. © Cerner Corporation All Rights Reserved 2 Agenda What are the objectives from a supplier perspective? Challenges of end-to-end work-flow testing Experiences from testing Cerner Millennium integration with PDS and C&B How could the testing experience be improved/made more efficient?

3. © Cerner Corporation All Rights Reserved 3 Objectives Integration Testing: testing a complete system in conjunction with national applications. LSP Contract Schedule 4.2 – Integration Testing Testing of end to end processes supported by the System functionality using real systems, including systems of the Integrated Service Providers, and involving all interfaces. Identify defects that are not easily identified during pre- integrated testing phases. Prove end-to-end workflows Gain “Authority to Proceed” certificate as part of movement towards live service.

4. © Cerner Corporation All Rights Reserved 4 National Systems tested Spine Services Spine Directory of Services (SDS) The SDS will be the definitive source of information on the following: Organisations, their Sites and Departments NHS Staff and Other Relevant Person Records Services Personal Demographic Service (PDS) - HL7 v3 The central source for patient demographic information, such as NHS number, names, addresses and date of birth Single Sign On (SSO) Uses NHS CRS Smartcards to help control who accesses the NHS CRS and what level of access that they can have Choose and Book (CAB) - HL7 v3 A national electronic referral service which gives patients a choice of place, date and time for their first outpatient appointment in a hospital or clinic. Patients can choose their hospital or clinic, and then book their appointment to see a specialist. Legacy NHS Numbers for Babies (NN4B) NSTS

5. © Cerner Corporation All Rights Reserved 5 National Integration Service (NIC) Processes

6. © Cerner Corporation All Rights Reserved 6 Testing phases System Testing Messages generated against harnesses for PDS, CAB and SSO The harnesses are applications that attempt to mimic the functionality of the Spine services PDS outbound messages validated using the Outbound Message Validation (OMVT) tool Messages and OMVT reports sent to the NIC for further validation Deployment into Integration Testing All Connectivity routes tested ITB Pipe cleaning Each message type tested Formal Integration Testing End to end workflows tested Functional and non-functional tests performed

7. © Cerner Corporation All Rights Reserved 7 Architectural design Interactions with the NASP use HL7 V3 message processing EbXML mode (asynchronous) Webservice mode (synchronous) SDS information is persisted via LDAP synchronization

8. © Cerner Corporation All Rights Reserved 8 Supplier Infrastructure

9. © Cerner Corporation All Rights Reserved 9 Messaging considerations Messages are triggered either from a user action or from a system process The mapping between the database and the HL7 message is done by Millennium interface engine All traffic transmitted across the N3 is encrypted using SSL HTTPS for HL7 messaging LDAPS for LDAP queries Every inbound/outbound message is archived for tracking and review purposes during testing Cerner implements a set of interactions as defined in NPfIT MIM (currently v3.1.09 for CAB and v3.1.10 for PDS). Higher versions of the MIM are supported in later releases.

10. © Cerner Corporation All Rights Reserved 10 PDS Messages Tested OUTBOUND from Cerner Millennium Advanced TraceQUPA_IN020000UK14 Simple TraceQUPA_IN010000UK13 RetrievalQUPA_IN040000UK14 NHS Number AllocationPRPA_IN040000UK15 General UpdatePRPA_IN110000UK15 Death Update PRPA_IN100000UK14 NHS ConfirmationQUPA_IN060000UK13 Consent UpdatePRPA_IN120000UK14 INBOUND to Cerner Millennium Simple/Advanced Trace Query SuccessfulQUPA_IN030000UK15 Retrieval Query SuccessfulQUPA_IN050000UK16 NHS Number Allocation Completed PRPA_IN060000UK14 General/Death/Consent Update SuccessfulPRPA_IN150000UK14 NHS Number ConfirmedQUPA_IN070000UK16 Query Act FailedQUPA_IN010000UK14

11. © Cerner Corporation All Rights Reserved 11 Choose and Book Messages Tested OUTBOUND from Cerner Millennium Notify Slot CancelPRSC_IN100000UK06 Notify Appt DNAPRSC_IN110000UK08 Notify Appt CancelPRSC_IN130000UK07 Respond Available SlotsPRSC_IN090000UK09 Confirm AppointmentPRSC_IN050000UK06 Reject AppointmentPRSC_IN060000UK06 Confirm Appt CancellationPRSC_IN140000UK06 Reject Appt CancellationPRSC_IN150000UK06 Application AckMCCI_IN010000UK13 INBOUND to Cerner Millennium Request Available Slots PRSC_IN080000UK07 Request Appt Confirmation PRSC_IN040000UK08 Request Appt Cancellation PRSC_IN070000UK08

12. © Cerner Corporation All Rights Reserved 12 Challenges of End to End Testing Technical Complex infrastructure – many potential failure points Complex dependencies e.g. must have SSO working before PDS can be tested Development work carried out against harnesses – not perfect representation of Spine services Interpretation of specifications Difficult to test component failure, latency, volume and performance. Procedural Co-ordination between different workgroups involved Limited previous experience with the functionality. Internal support organisation not familiar with issues. No established troubleshooting guides Writing test cases when some parts of design unclear. Writing test cases when requirements come from many different sources – only limited information in Bundle Map Requirements Making sure all configuration changes are properly documented and carried across into subsequent phases

13. © Cerner Corporation All Rights Reserved 13 NIC Assistance Review of Test plans and Specifications Endpoint/Service registration Test Data load into PDS Test readiness reviews On site and remote advice Helpdesk Daily update calls Review of Test Report

14. © Cerner Corporation All Rights Reserved 14 Experiences from testing Millennium integration with PDS and C&B Majority of NIC Helpdesk calls dealt with efficiently Most issues internal to solution and local infrastructure Relatively low number of issues caused by problems with sandpit/NIC Data: e.g. NHS Allocation loaded with invalid NHS number Backward compatibility issues with PDS response codes Unscheduled downtime 3 rd Party components – No clear release mechanism for Sun/Citrix SSO components Sandpit availability good Roadmap for upgrades and scheduled downtime given in advance Some flexibility dates for downtime Low number of C&B related issues. Relative maturity of solution Benefit of experience from Cerner C&B team

15. © Cerner Corporation All Rights Reserved 15 Possible Improvements Have a ‘technically conformant’ interface available during suppliers Module/Unit testing Closer involvement of design architects/software engineers in testing Testing conditions that need to be satisfied defined in design documentation Independent tool for viewing and modifying entries on the PDS Online issue logging and tracking Website showing real time spine service status

16. © Cerner Corporation All Rights Reserved 16 Questions Conan Ask cask@cerner.com 07903303182