1. Anne Sholander, MT(ASCP), International QA/QC Coordinator, SMILE
DAIDS clinical trials outside the US: Furthering the quality of laboratory operations The NIAID SMILE resource (November 2006)
Anne Sholander, MT(ASCP), International QA/QC Coordinator, SMILE

2. Who are the partners?
DCLOT
PPD
Non-Network
SMILE
Sites
Networks

3. How does DAIDS manage clinical laboratory oversight for HIV trials?
There are currently nearly 170 laboratories outside the US performing DAIDS funded HIV clinical trials.

4. Patient Safety Monitoring in International Laboratories
SMILE
Patient Safety Monitoring in International Laboratories
Our Mission:
To Provide Quality Assurance Support for
DAIDS-Funded HIV Clinical Trial
Laboratories (Non-US)
To Maintain a Web-Based Repository of
Records and Resources

5. “This contract will provide a critically needed comprehensive, uniform program to evaluate the capability of laboratories in developing countries, advise and train when deficiencies are identified, and ensure the ongoing quality of study test results and ultimately the outcome of clinical research performed in these countries.”
--original contract language
--outlined the goals of the project
--we remain true to these original goals

6. Clinical Laboratory Research Resources
We aim to achieve
Safety of trial subjects.
Safety of those performing tests.
Quality of the laboratory data.
Increased appreciation of labs for good laboratory practices that translates to better local service.
What are the goals of clinical laboratory research resources?
What do we hope to achieve with laboratory oversight?

7. Who is SMILE?
Contract resource between NIH/NIAID-DAIDS & Johns Hopkins University
Staff of 11 with a wide range of laboratory experience
Combined total of 185 years of experience
Wide variety of backgrounds and areas of expertise
we come from clinical labs, research labs, public health labs and from DAIDS funded HIV lab (Zimbabwe).
Depth of experience in variety of lab disciplines. SMILE staff is experienced in:
Microbiology/TB , Malaria
Chemistry/Hematology/Immunology
Validation of new methods, instrument troubleshooting
Manual differentials/urinalysis
Scientific writing and editing
We are especially fortunate to have a PI with a strong background in Bioinformatics. This has lead to development of extensive computer and web based tools used by SMILE in this project.

8. How does SMILE work to achieve the goals of our contract?
Primarily from our office in Baltimore.
Extensively through
Regular teleconferences with our site labs, networks and DAIDS.
Each lab is assigned a primary and a back-up coordinator.
regular (sometimes daily) communication with the labs
--develop a personal relationship with the labs.

9. A year in the life of a SMILE supported lab…
The best way to show you the tools that we use at SMILE is to walk through a typical year in the life of a SMILE supported lab.
How our tools work to achieve these goals.

10. Laboratory Audits
Independent assessments are performed annually by PPD or DAIDS
Based on the standards of Good Clinical Laboratory Practice (GCLP)
The “year” for the lab can be said to begin with the audit.
Independent of the SMILE contract—avoids any conflict of interest.
Audits performed by DAIDS contractor, PPD or by DAIDS.
PPD (or DAIDS) performs the audits and SMILE assists with remediation of audit findings.
Audits based on GCLP

11. Good Clinical Laboratory Practice (GCLP)
Organization and Personnel
Testing Facilities Operation
Verification of Performance Specifications
Quality Management
Equipment, Physical Facilities
Personnel Safety
What is GCLP?
US has a set of accepted guidelines and standards for clinical labs, there was a need for a unifying set of standards for international labs.
DAIDS AND contractor PPD developed GCLP guidelines.
GCLP are a set of standards based on FDA, CLIA, CAP, ISO, SANAs guidelines to ensure the quality of study data and allow for accurate reconstruction of experiments.
Some of the sections of the audit include:
Audit gives a comprehensive picture of the lab—and covers all aspects of Good Clinical Laboratory Practice. :

12. Audits and Action Plans
Audit report sent to SMILE
Action Plan is prepared
Networks designate critical items
Action Plan and audit sent to lab
What do we do with the audit information?
Audit reports sent to SMILE for review and preparation of an Action Plan.
Action plan contains the audit finding or observation, and SMILE prepares a suggested action for remediation of the finding.
Affiliated networks review Action Plan and designate items critical to their studies. This process also works for non-network affiliated labs.
Once the Action Plan is finalized it is sent to the lab along with the Audit.

13. The Action Plans are “Keyed” to the audit shell
The Action Plans are “Keyed” to the audit shell. Lab can cross-reference original audit report.
SMILE suggests an action for remediation and provides resources/training.
and we track the history of that action item.
The AP is color coded. Can see at a glance where the lab stands.
Generates feedback and interaction --between the lab and SMILE to resolve items.
SMILE is able to provide guidance and training based on our SMILE resources and experience of SMILE staff to address specific problems in the lab.
Used by networks and DAIDS as a tool to assess lab readiness for opening of new studies.
Also used as a tool to assess the readiness of a brand new lab to open it’s first study.
Goal is to close the Action Plan before the next audit.

14. SMILE Site Assistance and Training
Phone
Web conference
Adobe Connect
At conferences
On site
The Action Plan often leads to opportunities to help our labs by teaching or training.
Many times this takes place in or telephone conversations.
Our computer expertise and support has also allowed us to use web tools and create computer based training programs.
We utilize the Adobe Connect software which allows us to connect with our sites via the web and show them any presentations or walk them through any trainings that we have.
We have also presented trainings at conferences. TB Safety training was given at the last ACTG meeting and will be presented in Thailand in January.
We also are occasionally asked to visit a lab to provide specific training or assistance. Accomplish a specific goal or need—focused visit.
Our goal is to empower the labs through training. By doing this the knowledge and the experience stays with the lab even after the DAIDS study is finished.

15. Another type of lab assistance or training has been the SMILE TB lab observational visits.
SMILE has travelled extensively to assess and assist our labs with their TB testing capabilities.
Through these SMILE visits and the support of DAIDS and the cross-network TB working group, the quality, capacity, and safety of TB laboratories in developing countries has improved dramatically.
(We have already been to 17 network TB labs, 4 National Labs one of which is going to be added to network oversight and will have a PPD visit, 3 more will be in process during the presentation, 1 is scheduled for January.)
(This picture was taken in Uganda.)

16. Audits occur only once per year.
How do we monitor laboratory performance continuously throughout the year?
Back to the year in the life of our SMILE lab….and how we are managing laboratory oversight.
The lab has had an audit and is working on their action plan with SMILE. But the audit only occurs once per year.
How do we know throughout the year that the test results released from the lab are accurate?

17. Proficiency Testing
Proficiency Testing (PT) is a means of evaluating a laboratory's performance through analysis of unknown samples provided by an external source.

18. What is proficiency testing?
Also called PT or EQA (External Quality Assurance)
Blinded samples sent on a regular schedule
Run as patient samples
Results are compared to peers
The proficiency testing process is independent of SMILE—we cannot influence the outcome.
It is similar to audit where SMILE uses tool provided by an independent group to learn information about the lab’s quality.
SMILE provides assistance with ordering EQA.
Based on our experience with different providers and the requirements of the different network protocols, we are able to customize the PT order to the needs of the lab and the study.

19. Before we order EQA we need more information
Protocol analytes
Instrumentation in use
Back-up plans
Specimen flow
In order to be able to create a customized PT order for a laboratory, we need detailed information about the testing that will take place.
We first need to know which analytes will be needed for the protocol—both for safety testing and for study monitoring and endpoint data.
We also need to know which instruments and methods the lab will be using.
We need to know what the lab plans to do if their primary method of testing fails—what is their back-up plan.
It is also helpful to understand the specimen flow in the lab.
All of this information is captured on a Protocol Analyte List (or PAL) spreadsheet.
Protocol Analyte List (PAL)

20. This form was originally developed jointly by the networks/DAIDS and SMILE and it has become a critical tool for SMILE.
Specific to a particular protocol.
It shows the details of analytes are needed for a protocol
—and how and where they will be tested.
It also shows how we intend to cover those analytes with EQA—who the provider is and how frequently the panels will be delivered.
It is a way to communicate what the specific needs of a protocol are.

21. How does SMILE use proficiency testing?
Monthly review and monitoring of EQA results
Evaluation
Review
Tracking and summary
Reporting
Using the PAL, we have ordered EQA and the lab has begun testing the samples and submitting the results.
What does SMILE do with this information?
PT results are obtained from the EQA provider directly (usually).
PT provider has evaluated results.
SMILE provides the lab with a review of their survey results. Review process is a required element of GCLP.
Summarize the information and notify of investigations needed.
SMILE tracks PT performance of each lab on an EQA Summary Spreadsheet.
SMILE sends a monthly report to the lab and all affiliated networks (copying the DCLOT POC)detailing the performance on that month’s surveys and any pending investigations.

22. EQA Summary Schedule
On this first page of the spreadsheet is the year’s schedule for EQA shipments.
Lab knows a year in advance when to expect their survey shipments
This page also summarizes the laboratory’s status at a quick glace.
By color coding we can quickly see that there is an outstanding investigation required for Gram stain.

23. The second page of the EQA spreadsheet gives detailed information about each analyte tested in the lab.
The EQA history for 3 years is maintained on this sheet.
Color coding also gives a quick picture of how well the lab is performing.
So, for example, in 2008 this lab had difficulty with their platelet counts. This result would require an investigation—any result less than 100% successful (for a protocol analyte) requires a documented investigation.
In the case of this platelet count—the problem appears to have been corrected on the next survey.

24. Investigations Required for EQA failures
SMILE provides assistance, tools and resources
Root cause and prevention of future failure
Documentation of corrective action is end product
SMILE assists with investigations of EQA failure and provides labs with tools and resources for corrective action.
The process of troubleshooting the cause of EQA failure is another example of how important our extensive experience in the laboratory is.
The process often involves a lot of back and forth and research and discussion of possible causes.
Our goal is to teach the labs how to investigate problems and prevent them in the future by the use of good laboratory practices.

25. SMILE Investigation Form
The SMILE investigation form is designed to aid in the process of troubleshooting.
It identifies the analyte, and then asks a series of questions designed to lead the lab through a thorough investigation.

26. Root cause analysis
The goal of an investigation is root cause analysis and prevention of further failures.
We often help labs discover problems by prompting them to probe deeper into EQA failures.
We push them to find the cause of a failure and fix it, so that the integrity of patient testing is ensured.

27. Where do we store all of this documentation?
We are generating a lot of documentation
What do we do with all of this documentation—where is it kept?
One of our missions was to create a document repository.
We file all of each lab’s documentation on our website.
Psmile.org—this is our home page

28. Document Repository
EQA surveys, reviews and investigations on
Available to site labs (privately by log-in)
Available to DAIDS and the Networks
Also on
Audits and Action Plans
Site visit reports
Protocols and Protocol Analyte Lists (PAL)
Currently there are approximately 18,000 site lab documents stored on psmile and we upload an average of about 400 new documents each month.

29. Each site lab is updated at least monthly with their most current EQA summary, and new PT surveys and reviews.
Each lab can see only their own information—DAIDS and the affiliated networks also have access to this information.
Any investigations submitted during the month are also stored here. We also store the PPD Audits, action plans and PALs on psmile.
However, Psmile is much more than just a document storage website.

30. Resources on pSMILE.org
One of the things we are proudest of is our collection of resources posted on psmile.
SMILE resources are open to all—no log in is required.
This area of psmile is searchable by keyword. For example, if a lab is looking for information on Water Quality, they can simply type this in the search box.
There are currently ~1400 documents and weblinks posted in the resources section.
Many of these resources were written or developed by SMILE staff. We have devoted a considerable amount of our efforts to improving and perfecting our web resources.
Audit remediation, Training and Competency, Instrument validation
Laboratory move planning
GCLP compliance and Safety
Specific instrument/kit information
The success and evolution of our webpage is a credit to our PI Dr. Miller and his experience in the application of computer technology to the laboratory.
We are constantly encouraged to enhance and improve our use of computer and web-based resources.

31. Monthly reports sent DAIDS
Let’s recap the year…
Audit
Action Plan
EQA
Monthly reports sent DAIDS
1. The lab has had an audit
2. SMILE provides resources and works with the lab individually on their audit remediation by working to resolve action plan items.
3. SMILE reviews and evaluates EQA on a monthly basis.
SMILE is tracking the laboratory’s EQA and Action Plan progress and uploading all updates to
4. Monthly reports are sent to DAIDS that summarize all 170 SMILE labs

32. Summarizing the data EQA Exceptions Delinquent Investigations
Tracking Log
Nearly 170 labs-- how do we summarize the month for DAIDS ?
How do we keep DAIDS informed of what is going on with the labs, without overburdening them with the detailed information?
While we maintain all the detailed information, and it is available at any time—we also summarize the information on a monthly basis and send it to DAIDS.
The EQA Exceptions list shows labs that have two out of three unsuccessful rounds of EQA.
Alerts DAIDS and the networks that there may be a problem and testing may need to be moved to back-up.
Delinquent Investigations is a list of labs that are >60 days delinquent in submitting investigation.
Tracking log gives a brief summary of the month’s activities for each lab.

33. All of these activities create an ongoing dialogue between:
SMILE and the labs
SMILE and the networks
SMILE and PPD
SMILE and DAIDS
SMILE and IQA and VQA
All of these activities create an ongoing dialogue between:
This dialogue leads to constant, continuous improvement for our labs, for our processes, for our resources.
Although our focus at SMILE is on patient safety testing, we become more than just a resource for safety testing.
We also become a key contact and facilitator. We triage the questions to the appropriate organization.
We provide support to practically every test that appear in protocols, except for CD4 testing, where the IQA is responsible to monitor performance and for HIV virological tests where the VQA is responsible.

34. Additional benefits of SMILE
Experience
Discounted costs and improved services
Consistency
Flexibility
Collaboration
Experience—we have extensive experience in all aspects of laboratory oversight. We utilize every member of the staff and draw on his or her strengths to give the best possible service to our labs.
Discounts--Because of the number of labs we assist, we have an advantage in dealing with proficiency providers. We are the third largest user of CAP proficiency testing products, which gives us a tremendous advantage. We have points of contact and a relationship with the PT providers—we are able to work with them to solve problems unique to our labs. Ability to negotiate and manage premium shipping where needed.
Consistency across the SMILE staff so that all labs are treated in a uniform manner. Extensive internal SOPs, constantly refining the process and identifying areas for improvement.
Flexibility—we are not locked into the use of one PT provider, one shipper, etc. We have the flexibility to look at what will work best for the labs and customize their coverage accordingly.
Use of SMILE creates a sense of collaboration—between all affiliated networks, the labs and DAIDS. A unifying force that ensures that the requirements for each network and DAIDS are met and maintained in the lab.

35. In conclusion…
How does SMILE work to achieve the goals of Clinical Laboratory Research Resources?
To sum up—let’s look back at the original goals outlined for Clinical Laboratory Research Resources.
There were 4 goals outlined for laboratory oversight.
How does SMILE help achieve these goals?

36. Safety of trial subjects
We have an obligation and responsibility to protect the safety of the patients in our clinical trials
By ensuring that laboratory tests are accurate we can ensure the safety of trial subjects.

37. 2. Safety of those performing tests
Good Clinical Laboratory Practices place emphasis on laboratory safety.
Audit and Action Plan focus on safety of laboratory staff.

38. 3. Quality of the laboratory data.
Quality and integrity of study data.
Why conduct research if the data is not reliable?
Laboratory oversight ensures that study data is valid and reproducible.
Documentation is created that allows reconstruction of study experiments and shows corrective action for any problems that occur during the study.

39. 4. Increased appreciation of labs for good laboratory practices that translates to better local service.
When we improve the quality in the labs, it creates a ripple effect.
The culture of emphasis on quality spreads—to other labs, to clinics, to hospitals.
Education and job training for laboratorians improves.
Infrastructure and resources are improved.
All of this translates into improved quality of healthcare in developing countries.

40. Some of the things we are proud of…
Laboratories taking the initiative to improve their quality measures on their own
Laboratories striving for (and achieving) CAP, ISO and SANAs accreditation
Laboratories publishing/receiving awards
Recognition from local authorities of improved laboratory quality
Improved safety and increased testing capability and quality of TB diagnostic laboratories
Some of the things we are most proud of are the side-effects, or by products of our involvement with labs.
We strive to improve the lab as a whole—not to narrowly focus on only those things that directly affect our studies.
The SMILE approach to lab oversight reflects a shift in the paradigm, away from the old model of placing Westerners on the ground to manage the labs.
We aim for lab empowerment and improvement of local staff and infrastructure.
We are very careful to train, mentor, assist and encourage—without doing the work FOR the labs.
We want the quality of these labs to persist long after the DAIDS study has closed.

41. DAIDS clinical trials outside the U. S
DAIDS clinical trials outside the U.S. -furthering the quality of laboratory operations The NIAID SMILE resource
Dr. Robert E. Miller, M.D.—Principal Investigator
Kurt L. Michael, M. Ed., MT (ASCP)—SMILE Project Manager