Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The Quebec system of Monitoring and Enhancement of Physicians Performance André Jacques, MD, Director Practice Enhancement Division Collège des médecins du Québec Expect the Unexpected: Are We Clearly Prepared?

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Potential conflict of interests disclosure Dr André Jacques declare: “ I do not have any affiliation or financial interest of any nature in a business corporation, or I do not receive remuneration, royalties or research grants from a business corporation”

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Mission of the College The mission of the Collège des médecins du Québec is to promote a quality medicine in order to protect the public and to contribute to enhance the health of Quebecers

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Governance of the College Bureau (28 Governors) Executive Committee (5 Governors) President Professional Inspection Committee on Revision Committee on Discipline Committee on Admission to practice Committee on CPD Committee on Budget

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Management structure Legal Services Division External Affairs Division Administrative Services Division General Division Medical Education Division Practice Enhancement Division Inquiry Division

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Role of the College of physicians Admission to practice - registration –Accreditation of training programs and sites –Licensure examination for family physicians (written and oral examinations, OSCE and SOO) –Specialist certification examination (written and oral and/or clinical examination)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Role of the College of physicians Supervision of physicians’ practice –Assessment of practice of physicians in : Offices Hospital Nursing homes Etc. –Interventions to promote quality of practice CME activities Training programs Coaching Etc.

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Role of the College of physicians Investigation of complaints related to both physicians and illegal practitioners –Disciplinary process (for physicians) –Legal proceedings against illegal practitioners –Alternate resolution of complaints

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Role of the College of physicians Political role –Physicians and medical organizations –Public and media's –Government and official bodies –Other health professions

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Human and financial resources Staff of the College: –23 full time physicians –78 support staff Budget : –18.6 millions (Canadian dollars)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Quebec medical manpower Based on data of December 2005

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Quebec medical manpower Based on data of December 2005

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Code of ethics

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia A physician must practice his profession in accordance with the highest current standards; to that end he must improve and keep his knowledge up to date. (Section 44) Code of Ethics of physicians

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Professional Code

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia By-laws related to supervision of physicians practice A Professional Inspection Committee is established within each order. (Professional Code, section 109)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The committee shall supervise the practice of the profession by the members of the order. (Professional Code, section 112) By-laws related to supervision of physicians practice

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The committee may recommend that it require a member of the order to serve a period of refresher training or take a refresher course or both and that it restrict or suspend his right to engage in professional activities during the training period or course. (Professional Code, section 113) By-laws related to supervision of physicians practice

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia It is forbidden to hinder in any way a member of the committee, an inspector or an expert in the performance of the duties conferred upon him by this code. (Professional Code, section 114) By-laws related to supervision of physicians practice

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Professional Inspection Committee Functions Inquiries into the professional competence of a physician Inquiries into the the quality of the medical care provided in establishments

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Practice Enhancement Division

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Practice Enhancement Division Mandate The mandate of the Practice Enhancement Division of the College of physicians of Quebec is the monitoring of the practice of physicians and its enhancement in order to protect the public and contribute to enhance the health of Quebecers

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Practice Enhancement Division Human and Financial Resources Staff of the Division : –10 physicians 1 director 2 assistant directors 6 inspectors-investigators 1 physician for sick MD –1 coordinator –1 research agent –11 secretaries Budget : 4.3 M $

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Practice Enhancement Division Functions Evaluation of the quality of physicians' practice in office as well as in hospital Evaluation of the quality of the practice of medicine in health establishments Support to the Professional Inspection Committee

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Practice Enhancement Division Functions Intervene to enhance the physicians' performance in office as well as in hospital Support the Continuing Professional Development (CPD) Committee Maintain the secretariat of the Conseil québécois du développement professionnel continu des médecins

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Who will be the winner !

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Study of a representative sampling of 100 GPs Observational analysis of medical records randomly selected Assessment of some aspects of the quality of general practitioners’ professional practice Period :

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Study of a representative sampling of 100 GPs Professional inspection visits to 100 family physicians Random selection among Quebec’s family physicians : –practicing either entirely or partially in a private office –in the greater metropolitan Montreal region

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Study of a representative sampling of 100 GPs Two random sample groups : –52 family physicians licensed before 1980 –48 family physicians licensed between 1980 and 1996

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Study of a representative sampling of 100 GPs Professional inspection visits made by –5 inspectors/investigators from the College –Average of 10 years of experience in assessment of competence Study of at least 30 patients records randomly chosen Using of a standard grid listing the criteria, including a scores scale

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Criteria of evaluation Office practices Record-keeping Quality and quantity of CME Quality of practice –Investigation –Diagnosis accuracy –Treatment plan –Relevance of care

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Demographic data

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Demographic data

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Results Comparisons between physicians of groups 1 and 2 regarding : –Sex –Age –Status in hospital –University of issuance of the medical degree –Average number of consults per week

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Results No significant statistical difference detected between the two groups (except for record-keeping) No link between demographic variables and quality of care Strong link between : –record-keeping and quality of care –CME activities and quality of care

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Results

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Results

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Sampling of 100 GPs Conclusion of the study Results enable the College to reassure the public with regard to the overall quality of the professional practices of family physicians in Quebec Study confirms the PIC decision to evaluate the practice of physicians who present a potential risk of deviant profile, based on quality indicators

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Monitoring and enhancement of physician’s performance Screening of all physicians Monitoring Feedback to all physicians Enhancement CME programs for groups and/or individuals Assessment of physicians in need or at risk Remediation programs Detailed needs assessment MonitoringEnhancement Step 3 Step 2 Step 1

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Step

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Development of indicators for individual physicians Choice of indicator –By Professional Inspection Committee and Practice enhancement division, after consultation with different organizations Responsibility of the development –One staff physician

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Development of indicators for individual physicians Elaboration with : –Group of «experts» in the domain –Group of interested physicians in a hospital –Research group Pilot process evaluating feasibility Validation of the indicator Diffusion of the indicator with appropriate guidelines Implementation of the indicator

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The use of PSA levels as a means of detecting prostate cancer The use of PSA level as a means of detecting prostate cancer … When available data does not always allow to document practices and changes inducted by an intervention

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The use of PSA levels as a means of detecting prostate cancer Medical controversy about extended use of PSA as a means of detecting prostate cancer Publication of an evidence- based guideline

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The use of PSA levels as a means of detecting prostate cancer Publication of an information booklet for patients (in French) Local CME activity (in a single region) Analysis of physicians' perceptions : –90% say they are aware of it –76% agree with it –58% say it has influenced their practice

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The use of PSA levels as a means of detecting prostate cancer Evolution of prescription profiles in Québec and in the region where a continuing medical intervention was conducted After disseminationAfter intervention Dissemination of guideline

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The use of PSA levels as a means of detecting prostate cancer Data analysis … –Significant decreasing of the use of PSA levels as a means of detecting prostate cancer –Incomplete and impossible to analyze data Especially in Montreal area –Data to be collected locally by archivists (in patient records) Hard and long job

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Step

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Professional Inspection Committee Composition 2 physicians from the Practice Enhancement Division : The Director One Assistant director 6 physicians in practice, representative of both medical and surgical specialties 10 meetings per year

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Professional inspection visits Usual procedures Sending of a notice and a questionnaire to be completed before the visit by the physician or the establishment Inspection visit, with or without an expert Writing of a report and of a project of recommendations Submission of the report to the Professional Inspection Committee

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Professional inspection visits’ algorithm Recommendations Meeting with PIC secretary CME Syndic Administrative Committee PAM No control Control With expert Without expert Structured oral interview New recommendations Voluntary training Imposed training No limitation of practice Limitation of practice No controlControl

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Professional inspection visits’ algorithm Sending of recommendations Meeting with PIC secretary Transfert to Syndic Transfert to the PAMQ No control visitControl visitWith expert Without expert SOI New recommendations Volontary training programImposed training program Without limitation of practiceWith limitation of practice No control visitControl visit Good practiceUnsatisfactory practice Felicitations from the PIC Good practiceUnsatisfactory practice

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Individual inquiries 152 individual inquiries

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Results of inquiries Adequacy of :

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Decisions of PIC

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Practice enhancement activities Sending of recommendations from the Professional Inspection Committee Follow-up of recommendations by inspectors Organization of CME group activities (workshops on patient-doctor relationship, workshops on periodical health examination, courses)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Enhancement activities (n=533)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Workshops on Physician-patient relationship (n=296)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Workshops on Physician-patient relationship (n=296)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Other courses and workshops (n=125)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Step

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Detailed needs assessment Structured oral interview –Specific tool developed for GP’s –Individual evaluation of the performance –25 clinical cases to resolve –5 cases with simulated patients –Goal of the evaluation : personalized remedial CME prescription Evaluation training period –Used for needs assessment of specialists

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Structured oral interview Since 1990 : clinical cases –Assessment of knowledge In 1997 : addition of OSCE cases –Assessment of skills and attitudes In 2000 : addition of Script concordance tests –Assessment of clinical reasoning

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Structured oral interview Domains assessed «SOI» (20 cases) –Knowledge (data collection) –Problems solving –Interpretation of lab tests «OSCE» (5 cases) –Knowledge (data collection) –Physical examination (skills) –Interpretation of lab tests –Physician patient relationship Script concordance test –Knowledge (data collection) –Clinical reasoning –Interpretation of lab tests Content of the report Analysis of resolved cases Analysis of referred cases Detail of strengths and weaknesses Analysis of performance : –Emergency cases –Pharmaceutical treatments –Follow-up Interpretations –Responsibility level able to handle –Knowing of its limits –Inappropriate but not dangerous conducts –Harmful of dangerous conducts –Informed of recent conducts

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Structured oral interviews (n=101)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Structured Oral Interviews

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Individualized CME programs Clinical training programs Focused readings Tutorial chart recalls

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Individualized CME

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Personalized CME ( ) Statistics for all physicians (n=240)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Personalized CME ( ) Statistics for all physicians (n=240)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Personalized CME ( ) Statistics for all physicians (n=240)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Personalized CME ( ) Statistics for all physicians (n=240)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Personalized CME ( ) Training programs (n=225)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Personalized CME ( ) Results of remedial CME

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Monitoring and enhancement for improving performance in hospitals Screening of all health establishments Monitoring Comparative feedback to all health establishments Enhancement Feedback to the establishment Screening of establishments in need or at risk Meeting with authorities Control visit MonitoringEnhancement Step 3 Step 2 Step 1

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Step

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Monitoring of Health establishments Choosing of the indicators In collaboration with the concerned professional Association : –Selection of experts of the working group –Proposition of indicators by the working group –Evaluation of the feasibility by a pilot –Validation of indicators –Extension of the project to a group of hospital

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Monitoring with indicators Content of the pathological report of breast cancer

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Why choosing the pathology ? Request from the Professional Inspection Committee Interest of the Quebec's Pathologists Association for implementing a monitoring program with quality indicators

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia And why the breast cancer pathology ? Integral part of most of pathologists Media coverage of unhappy events in an hospital Launching of Provincial program of breast cancer screening

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Main steps of the program Fall 1998 : –Nomination of the experts' group Winter : –Completion of a pilot project –Final choice of quality indicators Summer 1999 : –Data collection (First study)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Main steps of the program Winter : –Analyze and evaluation of data by the experts' group Spring 2000 : –Feedback to hospitals Summer and Fall 2000 : –Enhancement activities by the Quebec’s Pathologists Association

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Main steps of the program Spring 2003 : –Data collection (Second study) Fall and winter : –New feedback to hospitals

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Experts' group Mandate : Identify quality indicators –Easy to monitor –Non expensive –Valid –Allowing to get a global view of the practice of pathologists

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Experts' group Composition : –The 5 members of the Quality-insurance Committee of Quebec's Pathologists Association –One member of College of Physicians' Practice Enhancement Division

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Evaluation of the last 20 medical records of breast cancer surgery in each of participating hospital Essential information for physicians in order to plan the treatment and establish the prognosis Quality indicators (Spring 1999)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Number of cases with research of hormonal receptors Indication - in the report - of dimension of the tumor Mention in the report of the distance between lesion and at least one resection margin Quality indicators (Spring 1999)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Mention of histological type in final diagnosis Mention of histological / nuclear grade Mention of the presence or absence of lymphatic or vascular invasion Mention of histological evaluation of resection margins Quality indicators (Spring 1999)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The first study (Summer 1999) Questionnaire sent to 53 hospitals Rate of answers : 100 % Number of records evaluated : 1040 (from the 53 hospitals) Number of pathologists aimed at the program : 247

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Results - Percentage of appearance of each element

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Feedback made by the Professional Committee in March 2000 at each of the 51 hospitals aimed at the program Feedback made to each hospital : –with tables and graphics –about results of the study for all hospitals aimed at the program –about their individual performance Feedback to hospitals

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Feedback to hospitals Histological evaluation of margins

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Feedback to hospitals Vascular / lymphatic invasion

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Feedback to hospitals Distance between lesion and margin

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Enhancement activities By Quebec's Pathologists Association –Production of a CD-ROM (prototype only) about breast cancer pathology –Presentation of the CD-ROM prototype at Quebec's Pathologists Association's annual meeting

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Enhancement activities By Quebec's Pathologists Association –Production and implementation of guidelines about elements to include in the pathological report of breast cancer in order to help physicians and surgeons to plan treatment and evaluate the prognosis.

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The second study Period of the study : Fall / winter Questionnaire sent to the 51 hospitals aimed at the first study Rate of answers : 100 %

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The second study Number of record evaluated : 999 –1 hospital excluded (not enough records) –1 hospital sent 19 records Number of pathologists aimed at the study : 246

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Results - Percentage of appearance of each element

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Compared results Rate of consistency per criteria

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Compared results Evaluation of lesion/margin distance

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Performance enhancement Evaluation of lesion/margin distance

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Step

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Visits in Health Establishments (n=24)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Activities of practice enhancement in health establishments Sending of recommendations by the Professional Inspection Committee and coaching by one of the College staff physician Organization of courses for answering specific needs Workshops on evaluation of quality given in health establishments

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Step

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Control visits with an expert In : –10 control visits of hospitals –1 control visit of a nursing home –1 control visit in another establishment

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Meetings with authorities in  On 3 occasions, meeting with administrative authorities of an establishment (1 hospital and 2 nursing homes)  On 8 occasions, meeting with the “Régie régionale” (in regard with 2 hospitals and 4 nursing homes)  On 4 occasions, meeting with Health Minister (in regard with 1 hospital and 3 nursing homes)

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia The Future !

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Competency Attitude Hability to practice Use of ressources Components of physician’s performance Professionnalism Patient TEAM

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia What are the conditions of success of that system?  4 key words who support that system: Commitment Investment Team work Partnership

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia First key word: Commitment  Commitment from my organization to the mandate of the Practice Enhancement Division:  Assessment and Remediation at all levels  More formative then punitive  Research agenda

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Second key word: Investment  Support from my organization to the mandate of the Practice Enhancement Division:  Budget from 860,000 $ to 4.3 M $  10 full time physicians dedicated to the mandate  1 research agent (4 days/week)  22 full time persons

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Third key word: Team work  Merging, in 1997, the CPD Division with the Peer Assessment Division to the new Practice Enhancement Division with a new mandate of assess physicians with a goal of remediate those in need.  Developing the new levels (level 1) to enhance a wider scope of physicians.

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Fourth key word: Partnership  This system cannot work without collaboration and partnership with:  The four Universities in Quebec  The two Unions (FMOQ-FMSQ)  The two others Colleges (RCPSC-CFPC)  The major medical associations

Presented at the 2006 CLEAR Annual Conference September Alexandria, Virginia Questions and Exchanges