1. GPC Methods Call PopMedNet Implementation Bob Greenlee 02/09/15

2. Overview What is PopMedNet Experiences with PopMedNet in other research networks – FDA Minisentinel – Others PCORNet Plans/GPC Status Next Steps

3. WHAT IS POPMEDNET

4. PopMedNet Open source, no-cost licensing software application for creating, operating, governing distributed health data networks Facilitates secure, customized, distributed analysis of electronic data – medical product safety, comparative effectiveness, quality, resource utilization, cost-effectiveness, etc. Much of the functionality is data model agnostic (i2b2 query integration possible) Architecture: Portal (one, central) + datamart clients (many, local) http://www.popmednet.org/

5. PopMedNet timeline (pre-PCORNet) http://www.popmednet.org/?page_id=7

6. EXPERIENCE WITH POPMEDNET IN OTHER RESEARCH NETWORKS

7. PopMedNet in the FDA Minisentinel Minisentinel: 5+ year development award to build a national system of post market medical product safety surveillance – 2009-2015. – 5-year Sentinel system awarded, will complete transition from MS to S ~ in late 2015 – Continues to grow and develop – Determination that MS/Sentinel is public health surveillance and outside jurisdiction of IRB/OHRP Minisentinel Distributed Database – Locally secure, distributed data within a standardized set of SAS tables in the MS Common Data Model format Tables: enrollment, demographics, dispensing, encounters, death, cause of death, lab results, vital signs No direct identifiers, but contains real dates, ages over 80 Developed in collaboration with MS data partners Borrows from HMO Research Network Virtual Data Warehouse model

8. PopMedNet in the FDA Minisentinel 4 main PopMedNet activities; all delivered through the MS PopMedNet datamart client – evolved collectively over the 5+ years Pre-refresh - central QA package to vet/approve site’s new CDM tables – 240+ output files Post-refresh - central programs run against CDM to – generates summary counts for MS Access database for use with automated ‘MS Distributed Query Tool’ – Generate ‘Common components’ – standard macro variables with site specific information (new) ~100 times per year – permit/review/release automated simple SQL queries using the Distributed Query Tool ~50 times per year – respond to analytic SAS programs to run against the full CDM – Originally many separately developed modular programs with different purposes/capabilities – Now being organized into the ‘MS Routine Query System’, with a master ‘Cohort Identification and Analysis’ program, and many accompanying options

9. Minisentinel – Simple Queries Simple Automated Queries – Prevalence Counts – number of events and members within a specified category Enrollment, drugs, diagnoses, procedures Restricted/stratified by age, sex, year, coverage, setting – Incidence Counts Same as above – Most frequent utilization – ranking top XX events within a specified prevalence category, stratified by age, sex, year, setting – Also supplies denominators 2 day turnaround – Can fully automate, or require approval at each step

10. MS Query - Email receipt

11. MS Query - Main Screen

12. MS Query - Request Details

13. MS Query - Output

14. Minisentinel – SAS program queries SAS programs to run against full CDM – Common analytic approaches in medical product safety surveillance Medical product exposures among individuals with/without conditions of interest Select events during exposure(s) to a drug/procedure group(s) of interest Health outcomes among individuals with or without conditions of interest Frequency and duration of treatment following an event of interest Drug use, diagnoses, and procedures before and after exposure of interest Drug use – uptake, persistence and patterns of use of newly approved drugs – Options for common observational research techniques Propensity Score Matching, Comorbidity score stratification, health care utilization stratification – Flexible inputs exposure, outcome, washout, temporal relationships, minimum supplies/durations, setting, diagnosis type, blackout periods, pre-existing conditions, lookback periods, query range, enrollment gaps, Charlson, utilization, rate ratio calculations, attrition – Usually summary results; 5+ day turnaround; ~50 per year

15. Minisentinel – SAS program queries MS datamart client does not yet execute SAS programs queries automatically against the CDM – delivered in zip package – requires minor local edits to adjust file input/output/reference table file paths – local execution against CDM tables – Package output into a zip and load back into datamart client for submission Ulitmately expect to run automatically against cdm from the datamart client, and will also obviate the need for summary count storage in Access for simple queries Automation has been done in the SPAN Network Scalable PArtnering Network (SPAN) for Comparative Effectiveness Research – Kaiser Permanente Colorado, AHRQ funded, ARRA program to develop a drn for cer – focus on adhd, obesity; 2010-2013 – 10 HMORN and nonHMORN partners Similar automatic SAS enhancement being developed within NCI Cancer Research Network’s CRNNet

16. PCORNET PLANS/GPC STATUS

17. PCORNet Plans - 1 DRN Query Tool V5.0 Release Timeline Final release to testing environment– Thursday, 12/11 Software migration scripts completed – Monday, 12/15 Migration tested and user testing complete with approval – Wednesday, 12/17 Release to PCORnet – week of 1/5 PCORnet v5 official release– Friday, 1/9 or Monday, 1/12* *As is always the case with software development, critical bugs or “blockers” may be uncovered during testing that could delay release. From pcornent steering committee dec 19, 2014

18. PCORNet Plans - 1 DRN Query Tool V5.0 Upgrade Steps Confirm network architecture with PopMedNet team (CDRNs) Establish network architecture within the Query Tool (PMN team) Re-installation the DataMartClient (CDRNs) Update DataMart profiles if necessary (CDRNs) Update DataMart metadata if necessary (CDRNs) This activity will begin 2 nd week of January and will continue on a rolling basis until each CDRN is completed. The process for each CDRN should only take a few days once their network architecture is finalized. From pcornent steering committee dec 19, 2014

19. PCORNet Plans - 2 Test the Querying and Governance Infrastructure Establish network architecture within the network Send simple query via the PCORnet Query Tool to test – Querying functionality (will the query run) – Network data release and approval process (can the results be shared) Query will generate simple frequencies based on real data in the demographic table (PCORnet CDM 1.0) From pcornent steering committee dec 19, 2014

20. PCORNet Plans - 3 Data Characterization Program Development Steps Develop functional specifications for data characterization program based on the Mini-Sentinel approach* (completed) Develop technical specifications for data characterization program (in process – expected completion January) Write and test data characterization program in one SQL flavor** (expected completion in February) Create a validation package with code, test database, and results to enable development of subsequent versions by one or more collaborators (February) Implement data characterization program in other SQL flavors (March) * www.mini-sentinel.org/data_activities/distributed_db_and_data/details.aspx?ID=131 ** CDRN store their data a variety of relational database system. Each system requires unique SQL code to execute successfully, therefore, the data characterization program must be written and tested in multiple “flavors” of SQL (eg, MYSQL, Oracle, teradata, Postgres, etc) From pcornent steering committee dec 19, 2014

21. PCORNet Plans - 3 Data Characterization Process Review and check all ETL ADDs for each site – For early adopters, discuss ETL ADD and findings from the query infrastructure testing prior to complete data characterization execution (to account for timing) Distribute complete data characterization code to each site via the PCORnet Query Tool – Basic checks of all data domains and data elements – Intended to assess consistency with the data model, characterize available data, and identify major issues for discussion Create data characterization report for each site Discuss data characterization report with each site, update ETL ADDs as necessary From pcornent steering committee dec 19, 2014

22. PCORNet Plans - 3 Data Characterization Process Considerations Note that the release of CDM V2.0 and 2.1 may overlap with the data extraction process for some sites. If this happens we would recommend that CDRNs use their CDM V2.0 or 2.1 for data characterization rather than their CDM V1.0. Therefore, it is possible that some CDRNs will undergo only 1 data characterization process using CDM V2.0 or 2.1. This would extend the data characterization timeline for those sites. From pcornent steering committee dec 19, 2014

23. PCORNet Plans – 4 PCORNet CDM V2 From cdm v2 stakeholder input presentation, january 23, 2015

24. PCORNet Plans – 4 PCORNet CDM V2 From cdm v2 stakeholder input presentation, january 23, 2015

25. PCORNet Plans – 4 PCORNet CDM V2 CDM Guiding Principles Most Pertinent with v2.0 Comments #2: It is not expected that all CDRNs and PPRNs will be able to populate all parts of the PCORnet CDM. It is the responsibility of the CDRNs and PPRNs to communicate availability of each data domain and element. #7: The CDM will reflect variables and values found in the local data. If some data are coded in a way that is unique to a site, mapping the data to a standardized format will be necessary. Values in the source data before mapping will also be included in the CDM. Derived variables should be avoided. From cdm v2 stakeholder input presentation, january 23, 2015

26. PCORNet Plans – 4 PCORNet CDM V2 #4: The PCORnet CDM will be developed in a modular, incremental, and extensible fashion. New types of data will be needed, or newly available, during the life of PCORnet. Data domains and data elements will be added, revised, and deprecated throughout an iterative CDM lifecycle. Personnel from the CDRNs and PPRNs will work with the DSSNI Task Force (and other Task Forces as appropriate) to assist in these efforts. #5: Documentation will be clear and transparent so that its contents are understandable to all contributors. The CDM will be intuitive and easy for analysts and investigators to use. Investigators and analysts with prior experience using research data will not need additional skills or knowledge to use the CDM. From cdm v2 stakeholder input presentation, january 23, 2015

27. GPC Status on CDM/PopMedNet GPC Call with DSSNI/Coord. Center on 1/13/15 – Bob, laurel, jim c, russ, phillip, susan, keith – GPC’s DSSNI liaison is Jessica Sturtevant from the CC at Harvard – Verifying popmednet contacts for each site – Clarifying database management systems at each site – Acknowledged our plan to set up a master data sharing agreement with coordinating center/designee site (external investigator agreement) to support a large portion of data characterization/quality assurance/feasibility queries Coord Ctr Developing Data Characterization WorkPlan Coordinating Center/Steering Commitee developing PCORNet governance documentation – Rolling out to CDRNs as they are ready, rolling out within CDRNs as they are ready – Periodic email contact from Jessica to our group Calls as needed

28. GPC Status on CDM/PopMedNet As of the first week of feb – 7 sites with the datamart client installed 2 on V5, 5 on V4 – Multiple sites have created V1 CDMs via i2b2 transformation using Nathan’s methodology – Basic sample queries being distributed Via Pitt, via CC through email (for informal local QA) Through datamart client (purpose???) – Multiple sites have begun running and comparing counts of individuals and facts in cdm tables with i2b2

29. NEXT STEPS (?)

30. GPC and PopMedNet Form PopMedNet Implementation Group – Periodic communications from DSSNI liaison/CC to all – Regular communications from Marshfield Datamart client nodes at each GPC site CDM transformation at each site More extensive quality/completeness checking of CDM – Additional quality/completeness scripts – MS CDM comparisons External data sharing agreement Respond to infrastructure testing query via the datamart client Data characterization process CDM v2 development Phase 2, centralization of GPC query response

31. CDRN Phase 2 RFA CDM issues Data infrastructure in place at time of application: full range of quality-checked data for core population of >= 1 million CDM V 2.1 or current Describe how individuals were included in core population, e.g., – Health plan enrollment – Certain number of visits – Other indicators to characterize defined population Reasons for missing data, plans to collect if feasible Activities to complete the capture of longitudinal data – Evidence of agreements with suitable partners in lieu of complete data (to be pursued under irb approved studies) Ability to execute queries against the CDM, including SAS queries that can run without modification. Must be in place at time of award. Return simple queries within 1 week of receipt Ability to continue development of CDM consistent with and beyond V2.1 (unstructured data, etc.) Policies and practices for security, privacy, and confidentiality Creation and characterization of 3 cohorts, document numbers available for query Enables computable phenotypes Facilitates patient level data linkage/sharing within the CDRN Refresh at least quarterly