1. © #OC15Europe 1 Electronically Collecting Data from Participants in Clinical Trials June 1, 2015 Ben Baumann bbaumann@openclincia.com

2. © #OC15Europe Trajectory  Terminology  Benefits & Promises  Barriers & Reality  Technology  Regulatory  Content  Data Management 2

3. © #OC15Europe 3

4. © eCOA (electronic clinical outcomes assessment) 4 Patient Driven eData Definition: an electronic assessment that may be influenced by human choices, judgment, or motivation and may support either direct or indirect evidence of treatment benefit. Assessment = instrument = questionnaire = form…

5. © #OC15Europe eCOA Modalities 5 ModalityDefinitionUse WhenExamples ePRO (Patient-reported outcomes) Data reported electronically directly verbatim by subjects. Self report, or interview. Trained health professional not needed + self-report feasible and appropriate. Unobservable concepts (feelings, sensations) Pain intensity Mood/feelings Eating habits ClinRO (Clinician-reported outcome) Report from a health-care professional observing subject’s condition. Involves clinical judgement/interpretation. Trained health professional needed tumor size Parkinson’s patient who cannot comment palpating a spleen ObsRO (Observer-reported outcome) Measurement based on an observation by someone other than the patient or a health professional. Does not include medical judgment/interpretation. Trained health professional not needed and self-report NOT feasible/appropriate. parent/infant caregiver/elderly person PerfO (Performance outcome) Performance of a task by subject based on instructions administered by a HC professional The specific task is required by protocol 6 minute walk test Memory recall

6. © #OC15Europe 6 Why do it?

7. © #OC15Europe 7 Better data Quality Faster, less labor Productivity Engage patients in new ways Enablement

8. © #OC15Europe May Clinic Study 8 How would you describe your overall quality of life today? 0 1 2 3 4 5 6 7 8 9 10 As bad as can be As good as can be  Score of 5 or less, 15 months survival  Sore of 5+, 52 months survival  Cannot ignore decrease of 2  3,704 cancer patients This is a prognostic factor for cancer patient survival

9. © #OC15Europe  Incorporated QOL assessment into daily clinical oncology practice  Initiated conversations with patients  Insomnia  Stupid thoughts 9  Reduced emergent care  Improved survival  Improved quality of life

10. © #OC15Europe Paper vs. Electronic Paper  Piles of paper can be confusing  Don’t need to worry about the internet  Chance of getting lost/destroyed  Slow access to actionable information  Parking lot syndrome 10 eCOA  More controlled experience  System prompts user automatically  Increased compliance (regulatory, patient)  Requires secure login/password  Date/Time Stamped in Audit Trail  E-Source recorded in Audit Trail

11. © #OC15Europe Drivers of eCOA  Quality & Efficiency  Increased data quality (edit checks, data types, show/hides, required fields)  More efficient data collection (reduce need for data entry)  Improved patient compliance 1,2  Auto-reminders  Electronically monitor  Know exactly when something happens  Report deviations to sites immediately  Get data earlier to prevent problems  Better patient care 11 Cost savings 1 ePRO often produces compliance rates over 90%. See Meltzer EO, Kelley N, Hovell MF (2008). "Randomized, Cross-Over Evaluation of Mobile Phone vs Paper Diary in Subjects with Mild to Moderate Persistent Asthma". "Randomized, Cross-Over Evaluation of Mobile Phone vs Paper Diary in Subjects with Mild to Moderate Persistent Asthma" 2 Friedberg F, Sohl SJ (2008). "Memory for Fatigue in Chronic Fatigue Syndrome: The Relation Between Weekly Recall and Momentary Ratings". International Journal of Behavioral Medicine 15 (1): 29–33

12. © #OC15Europe Drivers of eCOA (cont.)  Technology trends  Ubiquity of internet access  Mobile devices  Responsive design  Sensory devices  Apple ResearchKit 12

13. © #OC15Europe Drivers of eCOA (cont.) 13  Patient engagement  Integrate into subjects’ lives, especially if BYOD  Fewer dropouts  More events  Feedback  Improvement over time  Comparison vs. the field Engaged Patients Increased Compliance

14. © #OC15Europe eCOA in an ALCOA World 14 Attributable Legible Contemporaneous Original Accurate Login / access code Can control when things happen It’s digital, structured ePRO captured directly from subject Edit checks, rules logic Complete Consistent Enduring Available

15. © #OC15Europe 15

16. © #OC15Europe Barriers to Adopting eCOA  Expense (or, at least perceived expense)  Devices / Logistical complexity  Usability  Fit with study population  Integrating with existing EDC/CDM Systems  “Translating” existing PROs to electronic form  Others? What has stopped you? 16

17. © #OC15Europe Adoption of eCOA  2009 FDA PRO guidance 1  More trials are collecting COA data  Use of BYOD increasing, but mostly in post-approval studies  Scientific evidence growing to broaden use  Technology farther ahead of the regulations and adoption 17 1 http://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf

18. © #OC15Europe Success = Planning + Implementation So, let’s break it down: 18 Formula for Success Patient experience Content Regulations Technology

19. © #OC15Europe Patient Experience How will you engage your patients?  Motivation  Incentive  Feedback 19

20. © #OC15Europe Usability  Number of forms/instruments  Items per page  Scolling  Edit checks  Required fields  Skip logic  Accommodating disabilities  Offline  Support: patients, devices, logistics 20 “Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple. But it’s worth it in the end because once you get there, you can move mountains.” - Steve Jobs

21. © #OC15Europe Language 21  Will instrument be translated?  Avoid idiomatic expressions  Allowing for text length  Left Right, Right Left press click mouse select choose enter

22. © #OC15Europe Item Stems & Recall  Making questions self contained  During the last week, how has the pain interfered with:  Going to work  Sleeping at night 22  During the last week, how has the pain interfered with going to work?  During the last week, how has the pain interfered with sleeping at night?

23. © #OC15Europe Usability Testing  Field-test involving patients (ePRO) and/or clinicians (ClinRO) -- use the real thing 23

24. © #OC15Europe Electrifying Existing COAs 24 1 FDA PRO Guidance 2009: http://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdfhttp://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf  Must show eCOA version yields data which are equivalent or superior to paper version.  eCOA version must measure the same thing as paper  Need to show evidence of evidence of equivalence 1  Level of evidence depends on severity of change

25. © #OC15Europe Evidence of Equivalence 25 Table from ISPOR ePRO Task Force Paper: http://www.ispor.org/workpaper/patient_reported_outcomes/Coons.pdfhttp://www.ispor.org/workpaper/patient_reported_outcomes/Coons.pdf

26. © #OC15Europe Content: Leveraging Others’ Work  C-Path PRO and ePRO Consortia (www.c-path.org)www.c-path.org  Pre-competitive, FDA + industry  Collaborate to generate measurement equivalence data  Best practices  PROMIS (www.nihpromis.org/)www.nihpromis.org/  Freely distributes instruments  Tested, multiple languages  ISPOR (http://www.ispor.org/)http://www.ispor.org/ 26

27. © #OC15Europe Regulations: Available Guidance 27

28. © #OC15Europe 2009 FDA PRO Guidance How FDA reviews and evaluates existing, modified, or newly created patient-reported outcome (PRO) instruments used to support claims in approved medical product labeling. 28

29. © #OC15Europe 29

30. © #OC15Europe 2013 FDA E-Source Guidance  Assure reliability, quality, integrity, and traceability of data from electronic source to electronic regulatory submission.  Promotes capturing source data in electronic form  Retention of electronic source data in FDA-regulated clinical investigations. 30

31. © #OC15Europe 2013 Mobile Medical Applications 31

32. © #OC15Europe Mobile Medical Applications  Mobile App: A software application that can be run on a mobile platform, or a web-based software application that is tailored to a mobile platform but runs on a server.  Mobile Medical App: A mobile app that meets the definition of a device and is intended:  To be used as an accessory to a regulated medical device  To transform a mobile platform into a regulated medical device (e.g., via a sensor or electrode)  FDA intends to apply its regulatory oversight to only those mobile apps that are medical devices and whose functionality could pose a risk to a patient’s safety if the mobile app were to not function as intended. 32

33. © #OC15Europe Technology 33

34. © #OC15Europe Choices, choices… Web-based 34 ChoiceConsiderations Smart Phone Typically used for ePRO Well-integrated messaging (SMS, phone, email) Smaller screens may present issues for certain instruments Logistics if provisioning Tablets Typically used in ClinRO settings More readily emulate paper Logistics if provisioning Less validation burden Digital Pen Special pen, special paper Still have paper Legibility Technology support IVR Limited types of questions Recall time Patients get frustrated

35. © #OC15Europe App vs. Web  Limitations and Liberties  Pushing updates  Validation  Data stored on device  Responsive design  What actually is the app? 35 Web? App?

36. © #OC15Europe 36 BYOD? Provide device? Device Provisioning vs. BYOD DeviceBYOD User may forgetAlready in habit of using Greater controlUnknown device configurations Connectivity issuesAlready connected Simpler validationDevice checks Responsive design Connectivity issuesParticipate already connected Lots of logistics and support Little to none Hybrid BYOD / Provisioning an option …or both?

37. © #OC15Europe Role of Data Management  Edit checks/controls at entry  Required fields: complete data vs. accurate data  Frequency of data collection  EDC/CDMS integration  Reviewing data  How to handle data received too early / too late  Clear policy with sites around changing ePRO data (e.g. visit dates, basic demographics)  Under control of investigator at all times  Paper as back-up; how to handle  Site Support, FAQ 37

38. © #OC15Europe Putting it All Together 38

39. © #OC15Europe Thinking About ROI Paper Costs you are incurring:  Printing  Mailing  Data Entry  Data Cleaning 39 Electronic Costs you are incurring:  Electronic system Costs you are saving:  Less data entry  Less data cleaning  Printing and mailing  Fewer site payments Quality benefits:  Increased accuracy  Removing interpretation  More time for patients Faster study closeout Vs. Some estimates as high as $20 per page

40. © #OC15Europe We Can Do This! Technology has advanced and we have the power to use that technology to improve patient care. 40

41. © #OC15Europe Thank You! 41 Ben Baumann OpenClinica, LLC bbaumann@openclincia.com