Castine Clerkin, MS, CTR NAACCR VPR Program Manager Virtual Pooled Registry Cancer Linkage System (VPR-CLS) Pilot Testing and Next Steps Castine Clerkin, MS, CTR NAACCR VPR Program Manager Good morning everyone, I’m happy to lead off a series of presentations on the Virtual Pooled Registry initiative. My name is Castine Clerkin and I’m the VPR Program Manager at NAACCR and in this role, am involved in working with registries, researchers, key stakeholders, and IMS to develop a robust system for facilitating data linkages between researcher studies and registries around the U.S. Today I’ll be sharing information on our recent pilot test linkages and next steps. NAACCR/IACR Annual Conference June 12, 2019

Background: The Need and Limitations Many research studies rely on self-report of cancer and costly medical record validation Population-based registries offer researchers a standardized, comprehensive, and curated source of cancer data National linkages are ideal for many types of research studies Many limitations and barriers For those of you not familiar with the Virtual Pooled Registry I’ll provide a little background on the impetus for the VPR. Lots of research studies currently rely on self-report of cancer and costly medical record validation. As we all know population-based cancer registries are an existing source of standardized, comprehensive, curated cancer data and could therefore serve as an alternative way for these studies to ascertain cancer diagnoses or confirm self-reported cancers. For most types of research, including cohort studies, post-marketing surveillance, and clinical trials, national linkages are ideal; however there are various limitations and barriers that have stymied such national linkages with cancer registries.

Publication and data destruction State 1 State 2 Different Applications & Processes Meeting Renewal Requirements State 16 State 5 Help! Different Linkage Process (no matches!) State 15 Duplicative IRB Review State 6 State 7 State 14 Let’s look at the existing process, and associated barriers, for performing multi-registry linkages. Here’s our researcher, excited to be linking with registries to augment their study with cancer information. The PI wants to link with all U.S. registries (too many to show here!). The PI must approach every registry. Each has a different application and review process - some have both an IRB AND Registry application. The study is then subject to duplicative review across local IRBs sometimes resulting in local changes to the protocol. The registry and IRBs have varying review times, from a few weeks to over a year. The PI must monitor the status of the reviews and potentially respond to questions from each institution. Once approved, registries may require different formats for the study linkage file and use different linkage software and logic, possibly resulting in few, or no, matches after all that work. If the study is continues, they must meet renewal requirements and fulfill each registries requirements for publication and data destruction. As you can imagine, our poor researcher might be begging for help! Different Study File Formats Varying Review Times State 12 State 9 State 13 State 8 Monitoring Status of Review State 11 State 10

Virtual Pooled Registry Funded by NCI SEER and managed by NAACCR Designed to facilitate minimal risk linkage studies One-stop-shop for linkage application, file exchange, and match counts Streamlined process to apply for and track registry/IRB requests for release of individual-level data Minimize burden and cost to researchers and registries/IRBs Increase ease of access and timely use of high quality and complete registry data And to answer this call for help, NCI funded NAACCR to manage a virtual pooled registry that would facilitate minimal risk linkage studies through a one-stop shop for linkage application, review, and file exchange, as well as a streamlined process to apply for and track registry and IRB requests for release of individual-level data. The goal is to minimize the burden and cost to researchers, registries, and local IRBs while increasing access to and timely use of the incredible data that registries produce.

Virtual Pooled Registry Cancer Linkage System (VPR-CLS) Secure, online system to support linkages Developed by IMS, Inc. Linkage process and system development in two phases: Phase I: Initial linkage and provision of match counts by registry Phase II: Streamlined application process and tracking of requests for release of individual-level data on matched cases So just to be clear the Virtual Pooled Registry Cancer Linkage System (or VPR-CLS) is the secure, online system to support these linkages. The IT infrastructure is being developed by IMS and has proceeded in two phases. Phase I supports an initial linkage and provision of registry-specific match counts to the researcher. Phase II streamlines the application process and tracking of requests for release of individual-level data on matched cases.

Individual-level data on matched cases Phase II VPR-CLS Process: Streamlined application process for individual-level data on matches using: 1.) Templated IRB/Registry Application OR 2.) Central IRB Review System to monitor status of registry/IRB request, due dates, and requirements Phase I VPR-CLS Process: Phase I application Single study file Single linkage software and standard configuration Aggregate match counts States Selected Registry VPR Liaisons Linkage performed behind each registry’s firewall within 2 ½ weeks using an annual registry linkage file State 1 State 10 Here’s a basic depiction of how the process would work. Again, our happy researcher, submits an online request for the Phase 1 linkage with registries using a single study file, Match*Pro linkage software and a standard matching criteria. The cohort file is securely downloaded by each registry, who then performs the linkage behind their firewall within 2 ½ weeks using an annually created registry linkage file. The registries upload the resulting match counts. The researcher reviews results and selects registries for which to pursue the Phase II request for release of individual-level data on the matched cases. This application process is streamlined by the use of a Templated Application or Central IRB, which we’ll present on later. The system will monitor the approval status, along with renewal dates, and registry requirements for publication and data destruction. Once approved the individual-level data will be released directly to the researcher. One of the key pieces of the system is the registry VPR Liaison (represented here by my goofball children). The VPR Liaison is responsible for interfacing with the VPR-CLS, performing or ensuring timely linkages, and facilitating and updating the VPR-CLS on the status of the state review process. The Phase I functionality is in place and the Phase II functionality is in the final stretch of development. State 2 State 11 State 3 State 12 State 4 State 13 State 5 State 14 State 6 State 15 State 7 State 16 State 8 State 17 State 9 State 18

Pilot Linkages Before Online VPR-CLS Established Initial pilot linkages: ATSTR’s Camp Lejeune Incidence Study NCI’s US Radiologic Technologists Study Utilized Big Match software to test feasibility of standard linkage with 45 registries Linkage was feasible, but software didn’t meet VPR needs Prompted creation of Match*Pro

Pilot Linkages with 38 Registries Using VPR-CLS PHASE I ACTIVITIES Childhood Cancer Survivor Cohort Cancer Reg Follow-Up Transplant Cancer Match Lead Exp. Study Sister Study H.S. and Beyond Institutional IRB approval X Submit Phase I Application Upload Study Linkage File IMS File Review and Config Pre-test Linkages In process All Registry Linkages Match Count Reports So, we identified 38 cancer registries and 6 federally funded studies, to help pilot test the system and provide feedback. The six studies listed here will pilot test Phase I. The Phase I pilot linkage activities require that the PI obtain institutional IRB approval to link with registries through the VPR, submit their Phase I application, upload the study linkage file when approved, IMS then reviews the study file and creates a standard linkage configuration file to be used by all registries, a pre-test linkage then occurs at 4 registries to identify any issues before all registries are asked to link and upload the match count reports. The Childhood Cancer Survivor Study actually completed the pilot test last year. The remaining 5 pilot tests are at varying stages in the process. The Cohort Cancer Registry Follow-Up study has made it through the pre-test linkages, with the Transplant Cancer Match Study not far behind. Today I will summarize the linkage results for the Childhood Cancer Survivor Study and pre-test linkage results for the Cohort Cancer Registry Follow Up Study. When the Phase II functionality is ready, the CCSS and TCM Studies will collaborate on that testing. PHASE II PILOT TESTING

Test #1: Childhood Cancer Survivor Study Cohort of 36K childhood cancer survivors First or only primary diagnosed or treated from 1970-1999 Survived 5+ years from date of diagnosis Self report and medical record validation of subsequent neoplasms Linked with 34 registries (74.5% of U.S. population) 21 registries voluntarily performed manual review 75.6% sensitivity overall 6 false positives (0.12% of matches reviewed) So, the Childhood Cancer Survivor Study is a cohort of 36K childhood cancer survivors whose first or only primary was diagnosed or treated from 1970-1999 and survived more than 5 years from diagnosis. The study relies on self-report and intensive medical record validation of subsequent neoplasms. The study was linked with 34 registries – nearly 75% of the US population. During this process, 21 registries volunteered to perform manual review. Because this was an older cohort of children, many with missing SSN and changes in address or last name over time, the manual review resulting in a 75.6% sensitivity across registries. Only 6 false positives (or 0.12 % of the matches reviewed) were identified. Information on these false matches was shared with IMS to help adjust the Match*Pro linkage criteria.

Test 1: Match Count Results Match Count Reports by Diagnosis Year – no manual review All matches with cancer registry records Matches with same primary as study index cancer (within 2 years) The difference estimates matches with subsequent neoplasm (SN) Total Matches Index Match SN Match Self-Reported SN % Increase in SN Ascertainment 10,440 6,541 3,899 2,014 94% The researchers were provided with the registry-specific match counts by diagnosis year without manual review. While the match was person-based, both the study file and registry file contained site, histology, and date of diagnosis, so the linkage software was able to produce an initial report of the total number of matches and a second report that just identified the number that matched with the study index cancer (within 2 years). Using these two reports, we were able to estimate the matches that occurred with a subsequent neoplasm. As you can see here, there were 10,440 total matches across the registries, 6,541 were a match with the index cancer, and the remaining 3.899 were considered a match with a subsequent neoplasm. The researchers indicated that 2,014 subsequent neoplasms had been self-reported to the study. So linkage with these 34 registries has the potential to increase ascertainment of subsequent neoplasms by 94%, or nearly doubled the number of known subsequent neoplasms.

Test 2: Cohort Cancer Registry Follow Up Study Cohort of 290K individuals, includes participants from: Nurses’ Health Study I: 122K females, age 30-55, enrolled in 1976 Nurses’ Health Study II: 117K females, age 25-43, enrolled in 1989 Health Professionals Follow Up Study: 52K males, age 40-76, enrolled in 1986 Biennial questionnaires, including self-report of disease information The second study that I want to report on is the Cohort Cancer Registry Follow Up Study. This cohort of 290K includes individual’s who had enrolled in the 3 studies show here, which you may be familiar with. These female nurses and male health professionals respond to biennial questionnaires that included self-report of disease information like cancer.

Test 2: Pre-Test Linkage Pre-test linkage with 4 states (ID, GA, KY, NC) Match*Pro (V1.4): Reduction in volume of potential matches and improved accuracy of assignment to match/non-match Registries performed manual review of potential matches Minor modification to the Match*Pro configuration file This study has completed the pre-test linkage with ID, GA, KY, and NC. In this test, the latest version of Match*Pro was used and showed a noticeable reduction in the volume of potential matches and an improved accuracy of automated assignment to the match and non-match categories. Three of the registries were able to complete manual review prior to this conference. And only a minor modification to the Match*Pro configuration file was requested based on their review.

Test 2: Match*Pro Performance ID GA NC KY # of Registry Records 238K 992K 1.1M 590K Matches 197 612 1,662 709 Potential Matches 57 156 103 199 Matches after Manual Review 208 627 1686 713 Sensitivity 94.7% 97.6% 98.6% 99.4% False Positives So let’s look at the results and how well Match*Pro performed. Here you can see the number of records on the cohort file and the registry files, the number of matches automatically flagged by Match*Pro, and the number of potential matches that were manually reviewed (which was a really manageable number). After manual review you can see that Idaho added 11 matches, Georgia added 15, North Carolina added 24, and Kentucky added just 4. So the software’s automated assignment to true matches is performing really well. The resulting sensitivity was excellent and none of the registries identified any false positives.

Test 2: Anticipated Study Benefit of VPR Linkage All data limited to cases diagnosed 1995-2016 ID GA NC KY # of Study Self Report 160 583 1505 576 Initial Mathches (no review) 159 589 1560 670 % Study Ascertainment 100.6% 99.0% 96.5% 86.0% We also wanted to get a rough idea of how linkage with registries might improve the study’s ascertainment of cancer. The study manager sent me the number of cancers that have been self reported with a diagnosis year of 1995-2016 and residency in each of the 4 pre-test states. When these counts are compared to the initial registry matches (limited to the same diagnosis years) prior to manual review, you can see that study ascertainment is really high. Clearly these health professionals are much better at reporting a cancer diagnosis than the childhood cancer survivors.

Test 2: Anticipated Study Benefit of VPR Linkage All data limited to cases diagnosed 1995-2016 ID GA NC KY # of Study Confirmed 123 439 1056 412 Initial Matches (no review) 159 589 1560 670 % Study Confirmation 74.1% 72.7% 66.9% 61.5% The study manager also provided the number of self-reported cancers that the study had been able to confirm through medical record review. If we compare this number to the number of initial matches identified by the registry, we can see that linkage with the registries could significantly improve the degree of confirmation.

VPR-CLS Summary Registries and researchers are supportive of the VPR-CLS Pilot testing has successfully demonstrated the Phase I feasibility Match*Pro linkage software has been fine-tuned to minimize potential matches and eliminate false positives VPR-CLS process quickly supplies researchers with match counts in advance of completing registry/IRB applications Registry linkages can increase cancer ascertainment, confirm self-reported cancers, and augment study data for analysis However, until individual-level data released, the exact number of self-reported cases that matched the registry is unknown Phase I and Phase II pilot testing completed by year’s end

Contact: cclerkin@naaccr.org Questions? Contact: cclerkin@naaccr.org