Evaluating Competency Based Education in Clinical and Translational Science Wishwa Kapoor, MD, MPH

Definition of Competency Based Education (CBE) An educational approach which focuses on defining and measuring the outcomes of learning. CBE is a more applied approach to learning – demonstrating what the learner knows and is able to do. Educational objectives are defined through specific and measurable terms that assess the knowledge, skills, and attitudes (behaviors) learners are expected to have at the conclusion of study.

Background National Involvement – We participated in inaugural meeting in January 2008 sponsored by NCRR, to discuss the development of competencies in CTS. – NCRR then formed a national working group tasked with defining thematic areas and identifying core competencies within the educational structure for Master’s level clinician-scientists.

Background National Involvement – The working group developed a set of recommendations based on 14 themes defining the skills, attitudes and knowledge that should shape the educational experience of Master’s level clinician-scientists – Since 2008 the University of Pittsburgh has participated in the development of the CTSA/NCRR competencies and lead the group in the area of leadership and teamwork – University of Pittsburgh CTSI Education Core faculty participated in competencies discussions at 2008 and 2009 ACRT meetings

Development at the University of Pittsburgh Spring of 2008 CTSI Education Core Faculty: Group discussions about appropriateness and wording of competency themes Appointed a chair to lead a sub-committee for each theme Summer-Fall 2008 Sub-groups convened and used a modified Delphi method to finalize: Competencies for the theme Appropriateness of competencies for the Certificate, MS, and PhD Levels of expertise in each competency for Certificate, MS, and PhD

Development at the University of Pittsburgh October 2008 All subgroups members met for a ½ day retreat to discuss: Levels of competencies for PhD, MS, and Certificate Present themes, competencies, gaps in curriculum, and gaps in assessment Winter Course directors for the core curriculum reviewed competencies and reviewed each course to determine where each competency was addressed within the current curriculum

University of Pittsburgh Levels of Competence Goal: Classifying the Progression of Learning Introduced Basic exposure to a concept or topic. Acquiring knowledge and understanding of concepts. Mastery Evidence of a deeper understanding of concepts and topics that are part of clinical and translational research. Use of knowledge in research. Application and analysis of concepts. Transferring knowledge into action (skill). Expertise Evidence of a deep tacit understanding of concepts and topics that are part of clinical and translational research and applicable in their own content area. Synthesize, compare, and evaluate content.

NCRR ARRA Supplement Supplement will allow us to continue the development and refinement of competencies and move the entire educational offerings toward implementing and evaluating the outcomes of a competency-based educational (CBE) program

Specific Aims of Supplement 1.Refine and finalize a set of core and specialty track competencies for Certificate, MS, and PhD in Clinical and Translational Science. 2.Map competencies into currently offered courses. 3.Add new sessions to existing courses as needed, and develop new seminars, courses, experiential learning using innovative teaching strategies for a comprehensive CBE program.

Specific Aims of Supplement 4.Evaluate the CBE process using standardized assessment instruments. 5.Disseminate work to other CTS degree programs and present the work nationally and share with other CTSAs.

Aim 1 - Refine and finalize a set of core and specialty track competencies 1.Reassemble faculty competencies committee to review relevant literature as well as the work completed by the NCRR committee 2.Finalize competence themes 3.Articulate the expectations (knowledge, skills, judgment) of the certificate and degree programs 4.Refine competencies to reflect expectations 5.Revise and create performance levels that reflect the competencies established for all degree and certificate programs

Aim 2 - Map competencies into currently offered courses. 1.Review the content of each course to identify where competencies are taught, not taught, or need to be highlighted 2.Map competencies into courses where none exist 3.Develop a curriculum matrix which shows the location and the extent to which competencies are covered across all curricula

Aim 3 – Revise and develop new courses and teaching strategies 1.Experts in course design, teaching, and education will work with faculty to develop innovative teaching strategies to implement within the curriculum. Some examples of innovative teaching strategies include: Web-based technologies Multidisciplinary teaching Interactive learning methods Simulation methods

Aim 4 – Evaluate the process and the outcomes of implementing a CBE program 1.Assessments will be revised and created to reflect the standards and competencies established throughout the degree programs. 2.A formative evaluation will be conducted on the implementation of CBE in 3 areas: – Evaluation of the process of implementing competencies across certificate and degree programs – Evaluation of teaching and learning outcomes – Program Evaluation

Aim 5 – Disseminate work to other CTSAs 1.Through writing papers and presenting at regional and national meetings we will share our work on the process and outcomes of implementing and evaluating a CBE program. 2.Provide a complete curriculum mapped to competencies and evaluation. 3.Form a national group focused on CBE in CTS education.

Timeline of Project

Job Creation Program Coordinator – Full time program coordinator will be hired to handle everyday operations of the competency based education program. Senior Instructional Designer – Part time instructional designer will provide assistance in designing new courses, faculty teaching labs, seminars, and novel instructional media to implement within the CBE program.

Job Creation Instructional Technologist – Part time technologist to provide informational technology support for web-based applications. Graduate Student Researcher (GSR) – Full time GSR will help implement and evaluate competencies across curriculum.