Setting the Foundation of your CPD Accreditation Program Robin Kinch November 2018

Conflict of Interest Disclosure Robin Kinch I am a full-time employee of the Royal College of Physicians and Surgeons of Canada I have no relationships with commercial interests to disclose

Learning objectives At the end of this session, participants will be able to: Explain the criteria used to identify a physician organization; Discuss the minimum activity accreditation standards that must be met for accreditation; and, Identify at least one area for improvement in their current review process. This workshop has been developed to meet the following objectives: Participants will be able to explain the criteria used to identify a physician organization Participants will be able to discuss the minimum standards that must be met for accreditation Participants will be able to identify at least one area for improvement in their current review process

Introductions: Meet the RC CPD Accreditation Team! Julie Lavigne Manager CPD Accreditation Robin Kinch Program Coordinator Kate Runacres Program Coordinator Chanelle Goulet Administrative Coordinator Vacant Program Coordinator Provider Accreditation Provider Support National Standard International Accreditation NORs Self-Approval

The CPD Accreditation System We are All in This Together

Royal College Accredited CPD Providers Community of providers (your colleagues): 29 National Specialty Societies 3 Limited Accredited Providers 16 Canadian University Offices of Continuing Medical Education 16 Accredited Canadian Simulation Programs 3 Accredited International Simulation Programs The community of accredited CPD providers is comprised of national specialty societies, university offices of CME, accredited simulation programs, and other national and provincial organizations…

Maintenance of Certification Requirements MOC Program cycles are 5 years in length A minimum of 40 credits are required for each year A minimum of 400 credits are required to complete the 5-year cycle A minimum of 25 credits are required per section per cycle

CPD Activities: Section 1 – Group Learning Accredited Section 1 Activities Conferences, courses, seminars, workshops Web-based group learning Self-Approval Hospital rounds, tumor boards, M&M rounds, journal clubs, and small group learning

Comparison of Educational Standards Under the Educational Standards there is a difference between the standards for group learning activities and assessment activities. Section 1 Section 3 Educational Standard: Addressing Needs Educational Standard: Learning Objectives Educational Standard: Educational Format Educational Standard: Assessment Format Educational Standard: Interactivity Educational Standard: Process to record answers (SAPs only) Educational Standard: Feedback (SAPs and SIMs) Educational Standard: Evaluation Under Part 2: Educational Standards there is a difference between the standards for group learning activities and assessment activities. Group learning activities have an interactivity requirement whereas the assessment activities have an assessment/feedback requirement. We will explore these a little more as we go along.

Web-Based Activities

Section 1 Web-Based Group Learning Activities Web-based programs must: Provide an opportunity for interaction between participants and faculty; Allow participants to log on to the interactive component to claim credit under Section 1; provide evaluation feedback; and Track participant attendance and provide certificate after the participants have logged onto interactive component;

Assessment Activities

CPD Activities: Section 3 - Assessment Accredited Section 3 Activities Self-Assessment Programs Simulation Programs

Assessment Activities: What are They? Assessment activities must describe the methods that enable participants to demonstrate or apply knowledge, skills, clinical judgment or attitudes. Assessment activities can be developed as self-assessment programs or simulation programs This standard is applicable to Section 3 activities. It states that self-assessment programs must describe the methods that enable participants to demonstrate or apply knowledge, skills, clinical judgment or attitudes. Self-assessment programs must provide participants with a strategy to assess their knowledge, skills, clinical judgment and attitudes in comparison to an established scientific evidence base (clinical practice guidelines, meta-analysis or systematic review, etc). All self-assessment programs must use methods that enable participants to demonstrate these abilities across the key areas of the subject area, topic or problem(s). The selected format must also enable participants to review their current knowledge or skills in relation to current scientific evidence. Applies to Section 3

All Assessment-Based Activities Must be developed or co-developed by a physician; be planned to address the identified needs of the target audience with a specific subject area, topic or problem; include learning objectives that address the identified needs of the target audience; provide detailed feedback to participants on their performance and knowledge; and, provide participants with an opportunity to evaluate the accredited program.

Assessment: SAPs vs SIMs SAPs are knowledge assessments that determine how participants apply their knowledge and identify any educational gaps. SIMs are scenarios that mimic the doctor’s practice environment, where they are observed directly and are debriefed by observers.

Self-Assessment Programs Must provide participants with a strategy to assess their knowledge, skills, clinical judgment and attitudes in comparison to an established scientific evidence base; and identify which questions were answered correctly/incorrectly provide learners with references for correct answers

Simulation Activities Must provide a strategy to assess their knowledge, skills, and attitudes in comparison to established evidence (scientific or tacit); provide a description of the scenario, an assessment tool and a copy of the feedback form (if written). assess the appropriateness or relevance of the scenario (whether there was sufficient instruction time; practice time, etc.); allow participants to assess whether the feedback received was related to their performance in relation to the desired competencies, skills and/or attitudes;

What if you Could Not Participate in the Simulation? Observers of accredited simulation activities in a group learning setting can record their time in Section 1: accredited group learning.

How Long and How Many Credits? Section 1 Accreditation = up to 1 year accreditation period (1 credit per hour) Section 3 Accreditation = up to 3 years accreditation period (3 credits per hour)

How to Accredit A CPD Program

CPD Activity Accreditation In order for an activity to be accredited for the MOC program it must: be developed (or co-developed) by a physician organization; be reviewed by an accredited CPD Provider. meet the activity accreditation standards; and The role as an accredited CPD provider is to review activities using the Royal College standards. In order for an activity to be accredited for the MOC program, it must be developed (or co-developed) by a physician organization and reviewed by an accredited CPD Provider. Each accredited group learning activity must meet the standards for accredited group learning activities to be eligible for MOC Section 1 credits. These activities include conferences, courses, and workshops. This, however, does not include rounds, journal clubs, or small group learning activities, as these can be accredited under the “self-approval process” through the Royal College CPD Unit. Assessment activities must meet the standards for accredited assessment activities in order to recorded under Section 3 of the MOC Program. This includes accredited self-assessment programs and simulation-based activities. Applies to both Section 1 and Section 3

Criteria for Physician Organizations All accredited CPD activities must be developed or co-developed by a physician organization as defined by the Royal College. A physician organization is defined as a not-for-profit group of health professionals with a formal governance structure, accountable to and serving, among others, its specialist physician members through: * Continuing professional development; * Provision of health care; and/or * Research All accredited CPD activities must be developed or co-developed by a physician organization as defined by the Royal College. A not-for-profit group of health professionals with a formal governance structure, accountable to and serving, among others, its specialist physician members through: * Continuing professional development; * Provision of health care; and/or * Research Organizations that meet the Royal College definition of a physician organization can apply directly to an accredited CPD provider for accreditation. Determining if an applicant organization meets the definition of a physician organization is an important accreditation standard, but it can also be a little tricky at times. Don’t be afraid to reach out to us if you ever need help determining if an organization meets the definition. Applies to both Section 1 and Section 3

Physician and Non-Physician Organizations Examples of physician organizations: Examples of non-physician organizations: Departments within faculties of medicine Hospital departments or divisions Medical societies/associations/academies Physician research organizations Health authorities not linked to government agencies Canadian provincial medical regulatory authorities (MRAs) Disease-oriented patient advocacy organizations (Heart and Stroke Foundation) Government departments or agencies (Health Canada) Industry (pharmaceutical, medical device companies) Medical education or communications 'For-profit' on-line educators, publishing companies or simulation companies Small number of physicians working together to develop educational programming

The Physician Organization is Responsible for… receiving any financial and in-kind support. approving all decisions to ensure that the accreditation standards the CPD activity are met. paying expenses at an accredited activity. providing attendees with certificates of participation. maintaining attendance records for a five-year period. submitting an application to an accredited CPD provider. developing the CPD activity against established accreditation standards.

Benefits for Physician Organizations Allowed to submit a completed program to an accredited CPD provider The content does not need to be reviewed by an accredited CPD provider as the physician organization is responsible for ensuring that the content is free of commercial bias and meets the accreditation standards.

What about Non-Physician Organizations? Non-physician organizations must enter into a co-development relationship with a physician organization

What is co-development? Co-development is the process by which two or more organizations — at least one of which must be a physician organization — prospectively collaborate to develop and implement an accredited educational activity, learning resource or tool. However, physician organizations may also co-develop activities with others — including other physician organizations, non-physician organizations, or an accredited CPD provider. Applies to both Section 1 and Section 3

In a Co-Development Relationship the Physician Organization Must…. assume control over all aspects of the planning process as if they were planning the event independently. be involved in planning all aspects of the CPD activity with the non- physician organization from the beginning of the planning process. be represented on the scientific planning committee. Recognize the co-development of the CPD activity with a co-development statement along with the accreditation statement on all program materials For a program to be co-developed the physician organization must assign a rep to sit on the planning committee from the very start of development to ensure that the accreditation and ethical standards are complied with. Applies to both Section 1 and Section 3

Required Accreditation Documentation Needs assessment summary Conflict of interest disclosure template Activity budget Activity evaluation form Program/agenda/promotional brochures Written agreement between physician organization and sponsors. Minutes or notes from the planning committee Access to online modules (Section 1 and 3) Assessment sheet (Section 3 SAP) Feedback template (Section 3 SIM)

When Receiving an Application, Ask Yourself… Was the program developed by a physician organization? Was the application signed by the Chair of the scientific planning committee? Is this a co-developed program? Is all of the required documentation included?

Accreditation Standards Now that you’ve conducted your initial review for completeness, let’s move onto the specific standards.

CPD Planning Process Circle Convene a scientific planning committee based on the target audience Gather and review needs assessment information Create learning objectives based on needs assessment Choose the best educational format Recruit faculty Develop an evaluation strategy EVENT OCCURS Issue certificates to leaners Review the evaluation s When you review an application, you can see how the CPD planning circle echoes the structure of the accreditation application

Scientific Planning Committee Requirements All accredited CPD activities must have a scientific planning committee that is representative of the target audience. The physician organization is represented on the SPC Industry is not involved in any decisions related to program content Applies to both Section 1 and Section 3

Needs Assessment

Needs Assessment Needs assessments are a means of identifying perceived and unperceived needs of the target audience by collecting data and other information from a number of sources. Applies to both Section 1 and Section 3

Needs Assessments Must… be planned to address the identified needs of the target audience. identify either perceived or unperceived needs that address areas of knowledge, skills, and/or performance. describe the approach used to identify the educational needs of their target audience. Applies to both Section 1 and Section 3 A needs assessment of the target audience must be performed to identify areas of knowledge, skills, and/or performance that the CPD activity intends to address or improve. The needs assessments can identify either perceived or unperceived needs and should be used to inform: the development of learning objectives the identification of appropriate learning format the selection of relevant educational content the development of evaluation strategies.

Perceived or Unperceived Needs perceived needs assessments: personal requests Observations questionnaires/surveys focus groups evaluation data objective data (e.g. practice information, literature surveys, clinical practice guidelines and experience). unperceived needs assessments: Assessment of physician performance from hospitals provincial or national databases self-assessment programs chart reviews 360 degree assessments case scenarios practice audits and/or quality improvement activities CMPA data Implementation of clinical practice guidelines-data Performance and competency data Applies to both Section 1 and Section 3

When Reviewing an Application, Ask Yourself… Does the planning committee represent the target audience? Was the needs assessment conducted for the identified target audience? What gaps in knowledge or in performance were identified in the needs assessment? Do the results of the needs assessment inform the development of the learning objectives? What does the needs assessment tell you about the best learning format? When you are reviewing the application, have a look at the primary target audience – does the planning committee represent the target audience?

Learning Objectives

Learning Objectives Allow participants in a learning activity to understand what information or skills they will acquire during the session or event. For this standard, Learning objectives must clearly describe the intent of the educational activity, be written from the perspective of the learner, and express the expected outcomes determined by the planners and faculty. Learning objectives must be created for the overall CPD activity and each individual session. The learning objectives must be printed on the program, brochure and/or handout materials The identified learning needs of the target audience should be utilized in the creation/development of the learning objectives. It I all about showing the traceability between the identified needs and the learning objectives Supporting documentation can include the program and the needs assessment summary Applies to both Section 1 and Section 3

Learning Objectives Must… clearly describe the intent of the educational activity; be written from the perspective of the learner; and, express the expected outcomes determined by SPC and faculty. start with the sentence: At the end of this session, participants will be able to: For this standard, Learning objectives must clearly describe the intent of the educational activity, be written from the perspective of the learner, and express the expected outcomes determined by the planners and faculty. Learning objectives must be created for the overall CPD activity and each individual session. The learning objectives must be printed on the program, brochure and/or handout materials The identified learning needs of the target audience should be utilized in the creation/development of the learning objectives. It I all about showing the traceability between the identified needs and the learning objectives Supporting documentation can include the program and the needs assessment summary Applies to both Section 1 and Section 3

Learning Objectives Must Also Be… created for the overall CPD activity and each individual session; printed on the program, brochure and/or handout materials; made available to participants prior to the activity; and, incorporated into the evaluation strategy. Applies to both Section 1 and Section 3

When Reviewing an Application, Ask Yourself… Is there a defined link between the needs assessment and the learning objectives? Have learning objectives been developed for both the overall activity AND the individual sessions? Are the learning objectives written from the perspective of the learner by including action verbs describing what the participant will be able to do or the action they will be able to undertake? Have the objectives been communicated to participants in advance?

Learning Formats

Learning Formats To facilitate adult learning, a variety of educational methods that address the learning needs and objectives that you have identified should be used. No matter which methodologies are used in content delivery, incorporating interactivity is essential. Applies to both Section 1 and Section 3

Learning Formats Must… link to identified perceived and /or unperceived educational needs to established learning objectives; and, address CanMEDS Framework beyond the medical expert role. Applies to both Section 1 and Section 3

Examples of Learning Formats plenary sessions small-group sessions round tables workshops break-out sessions debates online programs (with interaction between participants and faculty) Applies to Section 1

When Reviewing an Application, Ask Yourself… What does the needs assessment tell you about the best learning format? Is there traceability between the learning objectives and the chosen educational format? Is there traceability between the needs assessment and the selection of topics and/or speakers? Applies to both Section 1 and Section 3

Interactivity

Interactivity At least 25% of the total education time must be allocated for interactive learning. Interactive learning helps participants to understand, translate and apply content to their specific practice contexts. This standard is applicable only to section 1 activities It states that at least 25 per cent of the total education time must be allocated for interactive learning Interactive learning can be promoted through question and answer periods, case discussions, skills training, etc., Interaction builds a relationship between (and among) participants and the faculty, contributes to a supportive learning atmosphere and enables speakers to determine the degree to which participants understand the content. Applies to Section 1 only

Interactivity Examples of Interactivity: Q&A periods Case discussions/debates Skills training/simulation Small group discussion/group work Think/pair/share

Section 1 Web-Base Group Learning Interactivity The following are some acceptable options to incorporate interactivity for accredited web-based group learning activities: discussion forums and chat groups teleconferencing and videoconferencing Twitter and email

When Reviewing the Program Ask Yourself: Is there a minimum of 25% interactivity? Do the selected learning formats allow for interactive learning? Are the opportunities for interaction communicated in the program? If web-based, is there a means to allow participants to log on to the interactive component? INCLUDE FOR WEB AND LIVE

Evaluation

Evaluation Evaluations provide feedback about whether the target audience’s needs were addressed, learning objectives were met and participants were engaged. A good evaluation strategy is extremely useful for planning and improving future events or sessions. Evaluations also promote participant reflection by gearing questions toward what participants learned and changes they are planning to make in their practice. Participants must be provided with an opportunity to evaluate individual activities and the overall series of activities. The evaluation system must: allow participants to identify whether the learning objectives were achieved provide opportunities for participants to identify the potential impact for their practice. ask participants to identify whether the content was balanced and free of commercial or other inappropriate bias. provide opportunities for participants to identify the potential impact of the CPD activity for their practice. Supporting documentation could include a copy of the evaluation form or tool Applies to Section 1 and Section 3

The Evaluation Strategy Must: Allow participants to identify whether the learning objectives were met; Provide opportunities for participants to identify the potential impact for their practice; Allow participants to identify CanMEDS professional competencies; Ask participants to identify whether the content was balanced and free of commercial Allow participants to suggest any topics that they would like to learn about Applies to Section 1 and 3

Evaluation Strategies May Also Include: an intent to measure improved patient performance; an intent to measure improved health care outcomes; and, an option for participants to receive feedback related to their learning. No matter which methodologies are used in content delivery, incorporating interactivity is essential. Applies to Section 1 and 3

Certificate of Attendance

Certificate of Attendance The certificate of attendance must include the following elements: The title of the activity; The date(s) the activity took place; The location of the activity (i.e. city, country, web-based); The total number of hours for which the activity is accredited; All applicable accreditation statements (include co-development statement when necessary); and, All accredited CPD providers must maintain attendance records for a minimum of 5 years. This standard states that the physician organization must maintain records of attendance for a 5 year period and provide confirmation of attendance to participants which includes the appropriate accreditation statement. Here are specific elements that must be included on the certificate…(activity title, name of physician organization, activity date, location, # of hours for which activity is accredited, all applicable accreditation statements) We have sample accreditation statements on our website for your reference. Please make sure that you are always using the most up-to-date version of the accreditation statements. Applies to both Section 1 and Section 3

Accreditation Statements Accreditation statements should be placed on all promotional materials once accreditation approval has been received Statements indicating that an activity is “pending approval” are strictly prohibited. Applies to both Section 1 and Section 3

When Reviewing the Application Ask Yourself… Are participants provided with an opportunity to evaluate both session specific and overall learning objectives? Are participants provided with an opportunity to provide details if they did detect an inappropriate bias? Are all of the required elements present on the certificate of attendance? JULIE

The activity is accredited. JULIE

Accredited CPD Provider Post-review Responsibilities After the accredited educational program takes place: provide the appropriate accreditation statement(s) to the applicant. notify the Royal College of the approval using the “Notice of Review (NOR)” form. ask for a list of attendees (for distribution of certificates of attendance and for credit validation purposes). ask for a summary of the evaluation for the activity JULIE

Ethical Standards

Content Development The physician organization(s) must have control over the topics, content and speakers selected for this activity. The physician organization(s) must assume responsibility for ensuring the scientific validity, objectivity and balance of the content of the activity. Participants or sponsors can suggest faculty or topics; however, they cannot direct the selection process and the final decision must reside with the physician organization. The process by which the topics, content and speaker(s) are selected for an educational activity must be in the control of the physician organization and planning committee at all times. Participants or sponsors can suggest faculty or topics; however, they cannot direct the selection process. The final decision must reside with the physician organization. All faculty, speakers, and planning committee members must complete Conflict of Interest Declaration forms irrespective of whether a direct conflict exists. Forms must be submitted to the planning committee prior to the start of the activity.  All conflicts must be disclosed to participants either verbally, displayed in writing on a slide at the beginning of a presentation or included in the written activity materials. The planning committee must have policies and procedures in place to manage identified conflicts of interest once they are disclosed.

Conflict of Interest (COI) COI declarations must be received from everyone who influenced content (e.g., all faculty, planning committee, moderators, authors and/or co-authors) irrespective of whether a direct conflict exists. All faculty, speakers, and planning committee members must complete Conflict of Interest Declaration forms irrespective of whether a direct conflict exists. Forms must be submitted to the planning committee prior to the start of the activity.  All conflicts must be disclosed to participants either verbally, displayed in writing on a slide at the beginning of a presentation or included in the written activity materials. The planning committee must have policies and procedures in place to manage identified conflicts of interest once they are disclosed.

Disclosure Process Conflict of interest (COI) declarations must be submitted to the planning committee prior to the start of the activity.  The planning committee must have a management process in place to address identified conflicts All conflicts must be disclosed to participants either verbally, displayed in writing on a slide at the beginning of a presentation or included in the written activity materials.

What to Disclose Relationships with for-profit and not-for-profit organizations over the previous 2 years Any direct financial payments including receipt of honoraria; Membership on advisory boards or speakers’ bureaus; Funded grants or clinical trials; Patents on a drug, product or device; and All other investments or relationships that could be seen by a reasonable, well-informed participant as having the potential to influence the content of the educational activity.

Managing Conflict of Interest The speaker can alter the focus of the talk to limit the areas where conflict of interest is significant. The topic selected could be changed, but the same speaker be used. peer review of the content to ensure that balance has been respected. if a significant conflict of interest cannot be managed the topic and speaker can be eliminated.

Sponsors may also provide “in-kind” support. Sponsorship All funds received in support of the CPD activity must be provided in the form of an educational grant payable to the physician organization. Sponsors may also provide “in-kind” support. The terms, conditions and purposes by which sponsorship is provided must be documented in a written agreement signed by the CPD provider organization, scientific planning committee and the sponsor. Sponsors may provide support for an activity in the form of an educational grant payable to the planning committee. Sponsors may also provide “in-kind” support which can include logistical support, goods or services to support the educational activities, learning resources or tools. Additional funds management responsibilities of the physician organization(s) include:   The distribution of funds to all faculty and speakers, including the payment of honoraria, travel, accommodations or hospitality. Ensuring that all hospitality and other in-kind arrangements are modest and paid for directly by the planning committee. Ensuring that sponsors are recognized in a location separate from the educational content.

The Agreement Should Include: whether sponsorship is financial or in-kind support; the dollar amount of financial support provided; the specific in-kind resources provided; the sponsor’s agreement to comply with the National Standard;

The Agreement Should Also Include: that the scientific planning committee is not be required to accept advice from a sponsor as a condition of receiving financial and in-kind support; that specific interests must have no direct or indirect influence on any aspect of the development, delivery or evaluation of an accredited CPD activity; how sponsorship will be recognized; and how sponsorship will be disclosed to participants.

Managing Commercial Promotion Drug or product advertisements must not appear in: locations where accredited CPD sessions are occurring; preliminary or final programs; brochures; slides; and/or Communications to participants.

Managing Commercial Promotion It is the responsibility of the planning committee and faculty to ensure consistent use of just generic names, or both generic and trade names. Therapeutic recommendations for the off-label use of medication must be declared.

Tagging Tagging is the linking or alignment of a sponsor’s name to a specific educational session within an accredited group learning activity. Tagging” is strictly prohibited.

Unaccredited Activities Unaccredited group learning activities must: not conflict with or compete with accredited group learning activities. not be listed or included within conference programs, brochures or schedules (preliminary or final). This standard pertains to Satellite Symposia (which is defined by the Royal College as unaccredited group learning activities developed independently by non-physician organizations). Satellite symposia may not: occur at times that would conflict with or compete with accredited group learning activities; and May not be listed or included within conference brochures or schedules of accredited group learning activities. Tagging (which is defined by the Royal College as the linking or alignment or a sponsor’s name to a specific educational session) is strictly prohibited.

When Reviewing an Application, Ask Yourself… Does the COI reflect the requirements for the National Standard? Do the funds flow through the physician organization? Is there an agreement in place between the physician organization and sponsors? Are there any drug or product ads in the activities material? Is there tagging within the program? Are unaccredited activities managed appropriately?

Your organization’s policies and procedures Royal College website Resources Your organization’s policies and procedures Royal College website (www.royalcollege.ca/cpdaccreditation) CPD Accreditation Unit staff (cpd-accreditation@royalcollege.ca) Julie Lavigne(jlavigne@royalcollege.ca) Robin Kinch (rkinch@royalcollege.ca) Kate Runacres (krunacres@royalcollege.ca) Chanelle Goulet (cgoulet@royalcollege.ca)

The Best Resource is Each Other Suggest creating a community of practice