PILOT TRANSLATIONAL AND CLINICAL STUDIES (PTC) Thomas Campbell, MD Professor of Medicine Director, Pilot Translational and Clinical Studies

CCTSI Organizational Structure

Specific Aims 2018-2023 Implement a comprehensive PTC Program to accelerate high quality collaborative CTR throughout the CCTSI partner affiliates Promote all disciplines of research Prioritize proposals that advance the science of translation Conduct rigorous, transparent peer review of PTC applications to ensure that only projects with the greatest potential impact are supported by the program Two-tiered multidisciplinary rigorous review committee to identify the applications that propose to answer important scientific questions using innovative technologies and novel research methods Establish and implement metrics that will be used to monitor the impact of the PTC on scientific discovery within the Institute Program impact monitored both with NCATS common metrics and with metrics developed to meet local needs

Aim 1: Pilot Programs Colorado (CO)-Pilot Program Mentored Faculty Awards ($30,000/award) Junior Faculty Awards ($30,000/award) CSU-CU Pilot Awards ($60,000/award) Child and Maternal Health (CMH) Pilot Program Mentored CMH Pilot Awards ($30,000) Junior Faculty Pilot Awards ($30,000) Community Engagement (CE) Pilot Program CE collaborative - joint research project (community-academic co-PIs) to produce preliminary data for future competitive grant applications ($60,000/award) Translational Methods (TM) Pilot Program TM collaborative - collaborations to develop new methods, processes and technologies ($20,000/award) Natural models - new collaborations to advance the study of spontaneous diseases in animals as models for human disease and human drug and device development. ($20,000/award)

Aim 2: Peer Review Solicitation of applications Advertised to all faculty and trainees at affiliate institutions and community organizations Letter of Intent helps us assemble appropriate expertise for review panels and allows program leadership to advise applicants Review of applications and decision process Initial review and score by primary and secondary reviewers Peer Review Panel operates similarly to an NIH Study Section PACT Pilot Grants Committee reviews all CE Pilot applications All applicants receive a written critique Prioritization of applications Executive Committee reviews report from PTC Co-Directors with recommendations for funding based on impact scores Applications with widely discrepant scores, border the funding range are discussed prior to final decision

Aim 3: Monitor Impact Evaluation and monitoring Metrics of success Timely completion of regulatory requirements for awardees Interim (6-month) and final (12-month) progress reports Annual follow-up for three years, using standardized forms by the Evaluation Program Metrics of success External funding and patents (return on investment) Publications/presentations Sustained collaboration among awarded investigators Common Metrics of the CTSA consortium # pilot projects expending funds since January 1, 2012 % pilot projects with at least one subsequent research award. Triennial Needs Assessment Surveys Members and non-members of CCTSI Assess benefits and obstacles encountered by applicants Feedback used to modify the PTC to better meet investigator needs

Return on Investment (2009-2014)

New Governance Structure Director Thomas Campbell, MD Translational Methods Natalie Serkova, PhD Colorado-Pilots Thomas Campbell, MD Sue VandeWoude, DVM Child & Maternal Health Pilots William Hay, MD Community Engagement Pilots Donald Nease, MD Montelle Tamez Program Coordinator Sandy Chalmers (Intermediate) Animated vertical organization chart Tip: The instructions below describe how to reproduce the exact organization chart and animation effects in the example above. In the animation procedures, the number of animation effects that require modification in the Animation Pane will vary if the chart contains a different number of managers and employees. To reproduce the SmartArt effects on this slide, do the following: On the Insert tab, in the Illustrations group, click SmartArt. In the Choose a SmartArt Graphic dialog box, in the left pane, click Hierarchy. In the Hierarchy pane, click Organization Chart (first row, first option from the left), and then click OK to insert the graphic into the slide. On the Home tab, in the Slides group, click Layout, and then click Blank. Select the graphic, and then click one of the arrows on the left border. In the Type your text here dialog box, in the text box for the first-level rectangle, enter Director (or the name of the director). In the second-level rectangle enter Assistant Director (or the name of the assistant director). In the text boxes for the third-level rectangles, enter Manager 1, Manager 2, and Manager 3 (or the names of the managers). On the slide, do the following to add more employee rectangles: Right-click the Manager 1 rectangle, point to Add Shape, and then click Add Shape Below. Repeat this process one more time for a total of three rectangles below Manager 1. Select the graphic, and then click one of the arrows on the left border. In the Type your text here dialog box, in the text boxes for the fourth-level rectangles, enter Employee (or the names of the employees). Right-click the Manager 3 rectangle, point to Add Shape, and then click Add Shape Below. Right-click the Manager 2 rectangle, point to Add Shape, and then click Add Shape Below. If you would like to modify the number of directors, managers, and employees in the chart to match the structure of your organization, do one or more of the following: To add a rectangle below another rectangle, right-click the higher-level rectangle, point to Add Shape, and then click Add Shape Below. To add a rectangle within a particular level, right-click one of the rectangles in that level, point to Add Shape, and then click Add Shape After or Add Shape Before. Select the graphic. Under SmartArt Tools, on the Format tab, click Size, and then do the following: To add text to new rectangles, select the graphic, and then click one of the arrows on the left border. In the Type your text here dialog box, enter text. To delete a rectangle, select the rectangle and then press DELETE. Under SmartArt Tools, on the Format tab, click Arrange, click Align, and then do the following: In the Width box, enter 8.79”. In the Height box, enter 6.55”. Click Align Center. Click Align Middle. Click Align to Slide. To reproduce the effects for the employee rectangles on this slide, do the following: On the slide, select the graphic. On the Home tab, in the Font group, select Gill Sans MT from the Font list, and then select 20 from the Font Size list. Press and hold SHIFT, and then select all of the rectangles in the slide. On one of the rectangles, on the middle of the left edge, drag the blue square adjustment handle slightly to the left to increase the width of all of the rectangles. Also in the Format Shape dialog box, click 3-D Format in the left pane, and then do the following in the 3-D Format pane under Bevel: Also in the Format Shape dialog box, click Shadow in the left pane. In the Shadow pane, click the button next to Presets , and then under Outer click Offset Center (second row, second option from the left). Press and hold SHIFT, and then select all of the rectangles in the graphic. On the Home tab, in the bottom right corner of the Drawing group, click the Format Shape dialog box launcher. In the Format Shape dialog box, click Line Color in the left pane. In the Line Color pane, select No line. Press and hold SHIFT, and then select all of the employee rectangles in the graphic. On the Home tab, in the bottom right corner of the Drawing group, click the Format Shape dialog box launcher. In the Format Shape dialog box, click Fill in the left pane, select Gradient fill in the Fill pane, and then do the following: Next to Top, in the Width box, enter 5 pt, and in the Height box, enter 5 pt. Click the button next to Top, and then click Cool Slant (first row, fourth option from the left). Click the button next to Direction, and then click Linear Down (first row, second option from the left). In the Type list, select Linear. Also under Gradient stops, customize the gradient stops as follows: Under Gradient stops, click Add gradient stop or Remove gradient stop until two stops appear in the slider. Select the first stop in the slider, and then do the following: Select the last stop in the slider, and then do the following: Click the button next to Color, and then under Theme Colors click Blue, Accent 1, Darker 50% (sixth row, fifth option from the left). In the Position box, enter 0%. Click the button next to Color, and then under Theme Colors click Blue, Accent 1, Darker 25% (fifth row, fifth option from the left). In the Position box, enter 100%. To reproduce the effects for the manager rectangles on this slide, do the following: Press and hold SHIFT, and then select all three manager rectangles in the graphic. On the Home tab, in the bottom right corner of the Drawing group, click the Format Shape dialog box launcher. In the Format Shape dialog box, click Fill in the left pane, select Gradient fill in the Fill pane, and then do the following: Click the button next to Color, and then under Theme Colors click Blue, Accent 1, Lighter 40% (fourth row, fifth option from the left). To reproduce the effects for the director rectangles on this slide, do the following: Press and hold SHIFT, and then the director and assistant director rectangles (first- and second-level rectangles) in the graphic. On the Home tab, in the bottom right corner of the Drawing group, click the Format Shape dialog box launcher. In the Format Shape dialog box, click Fill in the left pane, select Gradient fill in the Fill pane, and then do the following: Press and hold SHIFT, and then select the director and assistant director rectangles in the graphic. On the Home tab, in the Font group, click the button next to Font Color, and then click Dark Blue, Text 2 (first row, fourth option from the left). Click the button next to Color, and then under Theme Colors click Blue, Accent 1, Lighter 60% (third row, fifth option from the left). To reproduce the connecting line between Manager 2 and the last employee under Manager 1, do the following: On the Insert tab, in the Illustrations group, click Shapes, and then under Lines click Freeform (11th option from the left). On the slide, do the following to draw a freeform line: Click the second point approximately 0.5” to the right of the first point. Click the first point on the right edge of the bottom employee rectangle. Click the third point approximately 3.5” above the second point. On the slide, select the freeform line. On the Home tab, in the bottom right corner of the Drawing group, click the Format Shape dialog box launcher. In the Format Shape dialog box, in the left pane, click Line Color. In the Line Color pane, select Solid line, click the button next to Color, and then under Theme Colors click Blue, Accent 1 (first row, fifth option from the left). Double-click the fourth point on the left edge of the Manager 2 rectangle to finish the freeform line. Also in the Format Shape dialog box, in the left pane, click Line Style, and then do the following in the Line Style pane: In the Dash type list, select Round Dot (second option from the top). In the Width box, enter 2 pt. In the Join type list, select Round. In the Cap type list, select Round. To reproduce the animation effects on this slide, do the following: Also on the Animations tab, in the Animation group, click Effect Options, and then click Level One by One. Also on the Animations tab, in the Timing group, in the Duration box, enter 0.50. On the slide, select the graphic. On the Animations tab, in the Advanced Animation group, click add Animation, and then click More Entrance Effects. In the Add Entrance Effect dialog box, under Basic, click Peek In, and then click OK. Also on the Animations tab, in the Advanced Animation group, click Animation Pane. In the Animation Pane, press and hold CTRL, and then select the second, fourth, sixth, eighth, 10th, 12th, 14th, 16th, and 18th animation effects (peek-in effects). On the Animations tab, in the Animation group, click More, and then under Entrance click Fade. Also in the Animation Pane, select the first animation effect (peek-in effect). On the Animations tab, in the Timing group, in the Start list, select With Previous. In the Animation Pane, click the double arrow under the animation effect to expand the contents of the list of effects. In the Animation Pane, press and hold CTRL, and then select the fifth, seventh, and ninth animation effects (peek-in effects). On the Animations tab, in the Animation group, click Effect Options, and then click From Top. In the Animation Pane, select the third animation effect (peek-in effect). On the Animations tab, in the Animation group, click Effect Options, and then click From Right. Also on the Animation tab, in the Timing group, in the Duration box, enter 0.50 seconds. In the Animation Pane, press and hold CTRL, and then select the 17th and 19th animation effects (peek-in effects). On the Animations tab, in the Timing group, in the Start list, select With Previous. In the Animation Pane, press and hold CTRL, and then select the 11th, 12th, 13th, 14th, and 15th effects. On the Animations tab, in the Timing group, in the Start list, select With Previous. In the Animation Pane, press and hold CTRL, and then select the 11th, 13th, and 15th animation effects. On the Animations tab, in the Animation group, click Effect Options, and then click From Left. In the Animation Pane, select the 20th animation effect (wipe effect). On the Animations tab, in the Timing group, do the following: On the slide, select the freeform line. On the Animations tab, in the Advanced Animation group, click Add Animation, and then under Entrance click Wipe. Also on the Animations tab, in the Animation group, click Effect Option, and then click From Left. Also on the Animations tab, in the Animation group, click Effect Options, and then click From Bottom. In the Duration box, enter 0.50 seconds. In the Start list, select After Previous. In the Animation Pane, drag the 20th animation effect until it is 15th in the list of effects. To reproduce the background on this slide, do the following: Right-click the slide background area, and then click Format Background. In the Format Background dialog box, click Fill in the left pane, select Gradient fill in the Fill pane, and then do the following: Under Gradient stops, click Add gradient stop or Remove gradient stop until three stops appear in the slider. Click the button next to Direction, and then click Linear Down (first row, second option from the left) Select the next stop in the slider, and then do the following: Click the button next to Color, and then under Theme Colors click Black, Text 1 (first row, second option from the left). Click the button next to Color, and then under Theme Colors click Blue, Accent 1, Darker 50% (sixth row, fourth option from the left). In the Position box, enter 32%. Click the button next to Color, and then under Theme Colors click Blue, Accent 1, Lighter 80% (second row, fifth option from the left).

Accomplishments: combined 2017 CMH and Co-Pilot Cohorts Completed progress reports from 15/15 projects funded 4 completed project within the 12 months, 11 required no-cost extension due to: Subject recruitment delays Changes in key personnel IRB and other regulatory approval 1 has received follow-on research funding 10 plan to submit new grant applications (R01, K99, foundation) 3 manuscripts (published/under review); 6 meeting abstracts 11 plan to continue working with their collaborative partnerships in the future

Accomplishments: 2018 PTC Cohort Institutional Funded Pilots Awards made in May 2018 NIH Funded Pilots  Institutional Funded Pilots  Budget Number of Awards Number of awards Colorado Pilots CO-Pilot Mentored $60,000 2 $150,000   5 CO-Pilot Junior $30,000 1 $180,000   6 CO-Pilot Collaborative $120,000 $60,000   1 Child and Maternal Health Pilots CMH Mentored CMH Junior $30,000  Community Engagement Pilots CE Collaborative Translational Methods TM Collaborative $100,000 5 $20,000    Total $430,000 14 $500,000 16

Accomplishments: Solicitation and Review of Applications for 2019 Cohort Colorado-Pilots RFA released in July 2018, 94 applications were subsequently submitted (increase from 79 in 2018) Peer review conducted October-December 2018 Top 19 applications submitted to the Executive Committee on 12/20/2018 and approved for funding Award announcements will be made in January 2019 with expected funding start date of 5/1/2019. CMH Pilots 74 Letters of Intent resulting in 28 final applications received (increase from 23 in 2018) A panel of 62 reviewers with both senior and mid-career faculty (as a learning experience for the mid-career faculty who have not yet served on NIH panels) Top 5 applications were submitted on 12/20/2018 to the CCTSI EC approved, with funding expected to begin 5/1/2019

Accomplishments: Solicitation and Review of Applications for 2019 Cohort Translational Methods Pilots 69 Letters of Intent received in response to RFA released in October 2018 (13 LOIs from CSU faculty) Applications will be received and reviewed in Q1 2019 with the top applications initiating funding after 5/1/2019 CE Pilots - Leadership is working with the PACT (Partnership of Academicians and Communities for Translation) Council to restructure CE pilot grants program to better accommodate our current level of available pilot grant funding

National CTSA Consortium Involvement Novel methods developed by PTC projects will be shared across the CTSA consortium nationally Will share results and innovations related to natural animal models of human disease, where appropriate, with CTSA One Health Alliance (https://ctsaonehealth alliance.org/)

Response to EAC Critiques “Increase CU-CSU collaborations” Have increased visibility of PTC programs at CSU Increased applications from CSU for 2019 cohort: CSU investigators applied for 12 Mentored or Junior pilot awards (11 CO-Pilots and 1 CMH Pilot); increased from 5 in 2018 20 applications received for CU-CSU CO-Pilot; increased from 14 in 2018; 6 in 2017 13 CSU investigators submitted LOIs for TM-Pilot; increased from 8 in 2018 7 of 8 CU-CSU CO-Pilots (2017-2019) use novel animal models; 4 involve natural animal models of human disease

Questions for EAC (1) Should the PTC pilot programs support research in international settings? Does your CTSA or institution offer pilot programs for international research? Considerations: Institutional investment has help develop a site for clinical/educational experiences and research in Guatemala; other programs also established in Zimbabwe, Indonesia, Rwanda Currently no formal mechanisms for pilot projects in international research The CO-Pilot and CMH-Pilot programs have received requests from faculty to support international research pilot applications Per PAR 15-304, “Foreign components, as defined in the NIH Grants Policy Statement, are allowed” Concern that regulatory complexities for human subjects research at international locations could affect ability to complete a pilot project within the planned one year period

Questions for EAC (2) How can we improve mentorship experiences for CO-Pilot and CMH-Pilot Mentored awards? What metrics should we use to monitor mentoring outcomes? Considerations: Mentored pilots are a major focus of the PTC (76% of budget) Mentored pilot program has been very successful: 48% with follow-on funding and R.O.I. = 16.2 (2010-2014 period) A formal career development plan by the applicant is not currently required – mentors are required to outline a plan in their support letter. Should applications for mentored awards require a formal career development plan? Integration with Translational Workforce Development resources (CO- Mentor, Pre-K, Team LITeS) could improve pilot program outcomes. Should pilot applications be encouraged (required) to propose participation in one or more TWD programs?