1. Welcome back, Examiners! Washington State Quality Award Examiner Training 2010 Session 2

2. WSQA Examiner 2009 2 First things first! THANK YOU again! Our mission: Improving the way we live, learn, and work in Washington by helping organizations improve through the use of the Baldrige Criteria for Performance Excellence

3. WSQA Examiner 2009 3 Objectives For This Class Learn to evaluate a WSQA application through all stages of the assessment process Do this with confidence In yourself as an Examiner and In the assessment process

4. WSQA Examiner 2009 4 Once-around question Why do you think someone suggested you would make a good WSQA Examiner?

5. WSQA Examiner 2009 Agenda: Session 1 1.WSQA organization overview 2.Baldrige criteria overview 3.WSQA Assessment process 4.Online scorebook preview 5.Independent review, step-by-step Preparing Key Factors Process review Results review Mostly exercises Mostly presentation 5

6. WSQA Examiner 2009 Agenda: Session 2 6. Ethics, code of conduct 7.Consensus, step-by-step 8.Key Themes 9.Site visit preview 10.Final scorebook preparation 11.“Lite” vs. “Full” 12.Summary Mostly exercises Mostly presentation 6

7. 5. Ethics, Code of Conduct Objectives: 1.Understand the importance of ethics in this process 2.Interpret and apply the Code

8. WSQA Examiner 2009 8 Conflict of interest issues WSQA seeks to avoid conflict of interest to protect process integrity Conflicts are frequent! First step in assignment process: determine any conflicts

9. WSQA Examiner 2009 9 Code of Ethical Conduct Document is in your workbook Why are ethics of utmost importance to WSQA and this process?

10. WSQA Examiner 2009 10 Exercise 4: Code of Ethical Conduct Homework Lessons Learned from Home work Questions on Conflict of Interest

11. 6. Consensus, step-by-step Objectives: 1.Understand consensus stage activities, roles, responsibilities 2.Be able to develop valuable comments

12. WSQA Examiner 2009 12 Topics Consensus stage activities: Developing process and results comments Recommending scores Comment guidelines – do’s and don’t’s

13. Stages of the Assessment Process

14. WSQA Consensus: Step-by-step

17. WSQA Examiner 2009 17 Consensus preparation and call(s) Each Examiner responds with agreement, suggestions for changes, or disagreement, stating rationales Category Lead facilitates consensus Team Lead and/or PEPD mentor leads final scoring discussion Can take anywhere from 4-8 hours (depending on what?)

18. Writing Well Written Comments

19. WSQA Examiner 2009 19 How do good comments add value? Criteria Application content Organization Profile The most valuable Comments/Key Themes

20. WSQA Examiner 2009 20 The Structure of Well Written Comments Well written comments incorporate criteria language, ADLI/LeTCI, scoring language, and application language.

21. WSQA Examiner 2009 21 Well Written Comment Guidelines Category 1-6 Does the comment link to the Criteria? Does it consist of 2-3 grammatically correct sentences (about 1000 characters)? Does it use examples from the application that illustrate the OFI/Strength? Does it include a so what if it is an OFI? Is the statement non-judgmental and non-prescriptive? Does the comment link to the scoring guidelines?

22. WSQA Examiner 2009 22 Well Written Comment Guidelines Tie your comments to the basic, overall, or multiple level Criteria requirements that are most important to the applicant. Write process comments so they contain: a subject identified from the criteria, the application, or the scoring guidelines; verb(s) and requirements from the Criteria; examples from the application, and figure numbers, as appropriate. Use the evaluation factors (ADLI or LeTCI), and scoring language, to clearly articulate the areas of the applicant’s response to clarify the strength or provide insights that will help the applicant improve overall organizational effectiveness and capabilities. Draw linkages between an Item and the applicant's Organizational Profile. Ensure that the comment does not contradict other comments in the same or other Items or in the Key Themes Worksheet.

23. WSQA Examiner 2009 23 The Style of Well Written Comments Use active voice and present tense (e.g., 'completes' rather than 'is completed'). Use a polite, professional, and positive tone. Use vocabulary/phrasing from the Criteria and the Scoring Guidelines. Describe what is missing if something "is not clear". “Not clear” comments must be resolved if the applicant has a site visit. If they don’t do it based on site experience then say they don’t do it. In lieu of the applicant’s name, use such phrases like "the applicant" or "the organization" to refer to the applicant in the independent and consensus scorebooks. Use the applicant's name in final feedback reports. Use the applicant's language/terminology when appropriate.

24. WSQA Examiner 2009 24 “Don’ts” to avoid in comments 1.Avoid parroting 2.Avoid being prescriptive 3.Avoid going beyond the criteria

25. WSQA Examiner 2009 25 “Do’s” for well-written comments (refer to comment guideline in workbook)

26. WSQA Examiner 2009 26 Example Strength Comment The applicant identifies their core competencies as the participating dentist network; expert claims administration; extraordinary customer service and performance culture. A SWOT analysis is used in the annual strategic planning process to identify core competencies. Correlation analysis is performed on satisfaction survey data and is used to determine key drivers of customer satisfaction. Initiative action plans accentuate core competency strengths. Deployment is evidenced by the use of bi-annual CAG meetings, ensuring alignment with the mission and the integration of data for easy access across all departments and customer segments Red-Criteria Green-ADLI Blue-Application Pink-Key Factor

27. WSQA Examiner 2009 27 Well Written Comments Don’t…. Go beyond the requirements of the Criteria or assert your personal opinions. Be prescriptive by using "could", "should", or "would". Be judgmental by using terms such as "good", "bad", or "inadequate". Tell the applicant that they aren’t doing enough of something Comment on the applicant's style of writing or data presentation. Use jargon or acronyms unless they are used by the applicant Only “Parrot” the application or the Criteria. Provide only enough language to add clarity—seek to add value rather than restate information.

28. WSQA Examiner 2009 28 How Could This Comment Be Improved? Application language: Share Food uses a biennial 12 step Strategic Planning Process that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the “Current State” step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. The Strategic Planning Process was initiated in 1997 and restructured in 2004 as a result of benchmarking analysis of food banks and other nonprofit organizations, and feedback from the state award process. Comment: Share Food uses a biennial 12 step Strategic Planning Process (SPP) that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the Define step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. This is proof that the organization has a great systematic approach to the SPP and extensive cycles of learning in 1997, and 2004.

29. WSQA Examiner 2009 29 How Could This Comment Be Improved? Application language: The manager’s vision helps Our Store to maintain a strong focus on customer satisfaction. The manager has worked hard to instill the belief that leadership and responsibility should be present at all levels. This rewards everyone at Our Store with the responsibility of planning, carrying out, and evaluating their own work. Comment: Our Store is in the beginning stages of building an approach to setting a vision for customer satisfaction, this includes the manager working very hard to instill the beliefs in leadership and responsibility. Everyone in the store receives a reward and this will ensure that everyone will have ownership in the organization and the ability to evaluate their own work, which should improve performance overall.

30. WSQA Examiner 2009 30 How Could This Comment Be Improved? Application language: Our Hospital builds relationships with customers through a variety of methods including patient enrollment, which serves as an orientation to enabling services; Patient-Family Advisory Boards, which have recommended and implemented improvements to patient care; and high-visibility community health activities. Using patient surveys, conducted randomly after discharge, Our Hospital determines the issues that lead to poor relationship results and uses this information to implement new relationship-building processes. The Patient-Family Advisory Board is an example of the results of this process. Comment: Customer relationships are built through a variety of methods including Patient enrollment, Patient-Family Advisory Boards, and community health activities. Surveys are conducted randomly after patients are discharged. The Patient-Family Advisory Board is used to determine issues that lead to poor relationships and implements new relationship-building processes.

31. WSQA Examiner 2009 31 How Could This Comment Be Improved? Application language: Customer groups are identified annually as part of the Strategic Planning Process using Our Hospital’s vision/mission/values as a focus. By analyzing demographic data from multiple sources, the cross-location Service with Spirit Team (SWST) is able to identify the gaps, look at disparities, and identify potential customers. Deployment of this process through multiple cycles has resulted in the development of several services designed to meet the unmet needs of customers in the Our Hospital’s service area. Comment: Through the SPP, customer groups are annually identified using the MVV as a focus. The SWST uses demographic data to identify gaps, disparities and identify potential customers. This process has been deployed throughout the organization, and through multiple cycles of refinement the customers needs in the service area have been met.

32. WSQA Examiner 2009 32 Well Written OFI Comment Guidelines May be used to give credit to the applicant for some areas that HAVE been addressed “While the applicant does x, y, z, it is not evident that a, b, and c are performed.” May begin with a statement about what is missing: “There is no evidence that…” “There is no indication of how…” “It is unclear…” “It is not apparent that there is a …” Must tie to criteria language, (ADLI), and/or to Key Factors that the applicant has identified in the Organizational Profile Must include a “so what” (Why does the applicant care about this? Why would they take action?) Usually these come from Key Factors / Org Profile. The closer the ‘so what’ can be to identifying the applicant’s specific needs, the better.

33. WSQA Examiner 2009 33 Example OFI Comment Although cross functional teams are chartered and aligned with the SPP, there is no indication of processes used by the teams or otherwise by the applicant to design and innovate its work processes to meet key requirements. There is no indication of how design and innovation processes address opportunities for agility and improvements in cycle time and productivity. It is also unclear how the team work process improvement approach is evaluated for annual process changes or improvements. An annual evaluation of strategic team alignment and improvements may provide efficiency and effectiveness in the accomplishment of strategic objectives, initiatives, and action plans. Red-Criteria Green-ADLI Blue-Application Pink-Key Factor

34. WSQA Examiner 2009 34 How Could This Comment Be Improved? Application language: Share Food uses a biennial 12 step Strategic Planning Process that involves community leaders, volunteers, member agencies, and donors. Strategic challenges are identified in the “Current State” step which includes inputs from the SWOT analysis, environmental scan, and funding mandate review. The Strategic Planning Process was initiated in 1997 and restructured in 2004 as a result of benchmarking analysis of food banks and other nonprofit organizations, and feedback from the state award process. Comment: Although Share Food uses a biennial 12 step Strategic Planning Process (SPP) that involves community leaders, volunteers, member agencies, and donors, there is no evidence of how blind spots are identified, nor how time horizons are set and how the SPP addresses those horizons. If the applicant implemented a more extensive benchmarking process that identified blind spots and included time horizons, there may be more of an opportunity for this organization to have a more successful Strategic Planning Process.

35. WSQA Examiner 2009 35 How Could This Comment Be Improved? Application language: The manager’s vision helps Our Store to maintain a strong focus on customer satisfaction. The manager has worked hard to instill the belief that leadership and responsibility should be present at all levels. This rewards everyone at Our Store with the responsibility of planning, carrying out, and evaluating their own work. Comment: Although the manager has worked hard to instill the belief that leadership and responsibility should be present at all levels, there is no indication of a systematic approach, nor is there deployment at any level, no integration, nor are there any cycles of learning.

36. WSQA Examiner 2009 36 How Could This Comment Be Improved? Application language: Our Hospital builds relationships with customers through a variety of methods including patient enrollment, which serves as an orientation to enabling services; Patient-Family Advisory Boards, which have recommended and implemented improvements to patient care; and high-visibility community health activities. Using patient surveys, conducted randomly after discharge, Our Hospital determines the issues that lead to poor relationship results and uses this information to implement new relationship-building processes. The Patient-Family Advisory Board is an example of the results of this process. Comment: There is no systematic approach to building customer relationships, or in conducting patient surveys. And although there is a Patient- Family Advisory Board there is no approach described for how issues are determined and how new relationship-building processes are developed.

37. WSQA Examiner 2009 37 How Could This Comment Be Improved? Application language: Customer groups are identified annually as part of the Strategic Planning Process using Our Hospital’s vision/mission/values as a focus. By analyzing demographic data from multiple sources, the cross-location Service with Spirit Team (SWST) is able to identify the gaps, look at disparities, and identify potential customers. Deployment of this process through multiple cycles has resulted in the development of several services designed to meet the unmet needs of customers in the Our Hospital’s service area. Comment: Although customer groups are identified annually in the SPP, and the Spirit Team (SWST) identifies gaps, disparities, and identifies potential customers, there is no indication how this information is deployed throughout the organization and how the information is used to meet the unmet needs of customers in the area.

38. WSQA Examiner 2009 38 Comment Writing Exercise Each person at the table select an independent review item (Cat 1-6 strength or OFI) Write a well written comment Swap scorebooks What feedback would you give to this examiner Use the comment checklist for examples Practice giving feedback to each other Report out the most significant key learnings (If time swap with to another person at the table)

39. WSQA Examiner 2009 39 Category 7 Comment Guidelines Category 7 Are specific results (numbers) used rather than generalities? Have trends, comparisons and/or appropriate benchmarks been considered? Does the comment include figure numbers? Have we summarized multiple charts with a consistent theme (trends, missing, comparisons, etc.)

40. WSQA Examiner 2009 40 (Well Written Result Comment Example There is a lack of comparative data in a number of the Results sub-categories; as an example, no comparative data is available for measures such as Discharge Weight of ELBW Infants (7.1-4), Improvement % of ELBW Infants (Figure Figure 7.1- 5), % of Breast Milk Available (Figure Figure 7.1-6), Outpatient Lactation (7.1-7) and Catheter Related Bloodstream Infections (7.1-8), Survival to Discharge by Rhythm (Figure Figure 7.1-17), and Well Child Visits Figure (7.1-19). The lack of comparative results may impact the value of Excellence - setting and achieving the highest standards.

41. WSQA Examiner 2009 41 How Could This Comment Be Improved? The applicant’s market share results for key service lines is strong, exceeding all comparative data across Discharges; Inpatient Surgeries; the applicant's Heart Services; and Neurosurgery Market. The applicant enjoys market dominance in these service lines and has shifted its focus to adolescent care and services to new geographic markets. The applicant has received numerous recognitions and awards for their leadership in these markets and service lines. These successes are important to their goal of achieving national recognition as a premier hospital, a tertiary and quaternary hub, and preferred provider of regional health care services.

42. WSQA Examiner 2009 42 How Could These Comments Be Improved? The applicant's Net Income from Operations (7.3.8) is above the comparison median and trending up over the past three years and overall applicant’s turnover rate has been decreasing. The applicant’s Net Sales between 2006 and 2008, have increased by 10% in 2006 to over 12.2% in 2008. The applicant is consistently above the goal of 9% for all three years, but is below the Baldrige comparison of 15% for all three years. Although the turnover rate for RNs has been increasing between 2006 and 2008. The turnover rate for RNs was 7.7% in 2006 and 10.3% in 2008. There have been substantial improvements in physician satisfaction from 13% to over 25% during the same time period.

43. WSQA Examiner 2009 43 How Could These Comments Be Improved? The applicant's Net Income from Operations (7.3.8) is below the comparison median and trending down over the past three years and overall applicant’s turnover rate has been increasing between 2006 and 2008, from 10% in 2006 to 12.2% in 2008. The turnover rate for RNs has been increasing between 2006 and 2008. The turnover rate for RNs was 7.7% in 2006 and 10.3% in 2008.

44. WSQA Examiner 2009 44 Comment Writing Exercise Each person at the table select an independent review item (Cat 7 strength or OFI) Write a well written comment Swap scorebooks What feedback would you give to this examiner Use the comment checklist for examples Practice giving feedback to each other Report out the most significant key learnings (If time swap with to another person at the table)

45. WSQA Examiner 2009 45 Consensus and conclusion If using online tool during consensus, changes may be made in real time Category Lead notifies Team Lead and Scorebook Editor when done Scorebook Editor compiles consensus scorebook with final edits, notifies WSQA Consensus scorebook should be finalized ASAP

46. WSQA Examiner 2009 46 Comment, scoring considerations “Benefit of the doubt” Give credit for what’s in the application Don’t penalize for incidental exceptions Not every process must show complete “DLI” Not every results example must include “TCI”

47. WSQA Examiner 2009 47 Comment, scoring considerations For LITE, only one score for entire results Category 7 Propose a score for results related to your categories Be prepared to reach team consensus for all of Category 7 Using scoring language can help the applicant understand comments and score

48. WSQA Examiner 2009 48 Category feedback Check and provide feedback on whether Cat. Lead’s comments: Include criteria requirements Provide meaningful feedback Address ADLI or LeTCI OFI’s provide relevant “so what” (at least some of them) Results cite figure numbers/titles and specifics References to application are correct Recommended score reflects comments in content and balance (strengths:OFI’s ratio) Back-up does this for full, team does this for lite

49. WSQA Examiner 2009 49 Information resources WSQA comment guideline handout Your back-up, team lead, PEPD mentor Reference materials in Your workbook: comment and scorebook examples http://www.wsqa.net/extraining.php Baldrige web site http://baldrige.nist.gov/Examiner_Resources.htm http://baldrige.nist.gov/Examiner_Resources.htm

50. 7. Key themes Objectives: 1.Be able to identify Key Themes 2.Be able to develop a Key Theme comment

51. WSQA Examiner 2009 51 How do good comments add value? Criteria Application content Organization Profile The most valuable Comments/Key Themes

52. WSQA Examiner 2009 52 What is a Key Theme? A “key theme” is a strength or OFI that Addresses a central requirement of the Criteria, Is common to more than one Item or Category (crosscutting), Is especially significant in terms of the applicant’s key factors, and/or Addresses a Core Value of the Criteria

53. WSQA Examiner 2009 53 Developing Key Theme comments Address all double strengths and OFI’s individually and/or in groups Address related comments across categories Develop 6-10 (Guideline!) key themes each for significant Process strengths Process OFI’s Results strengths Results OFI’s See example Key Themes in the workbook for this class

54. Key Theme Development using the Baldrige Core Values  Purpose: Create Key Themes at the end of the consensus meeting or at the end of a site visit WHY? to identify Key Themes that cut across categories and items

55. WSQA Examiner 2009 What is a Key Theme?  A high level perception or observation, relevant to the applicant’s key factors, that recurs in the scorebook and is expanded upon with evidence during a site visit. Key themes are often common to more than one category or item. They may address one or more Baldrige core value.  A role model result, practice, process, or innovation, that might be used by other organizations, or an observation so important, strength or OFI, process or result, that it should be brought to leadership’s attention.  The executive summary of the feedback report. It may be the only part of the feedback report that some people in the organization read.

56. WSQA Examiner 2009 Cross Cutting Themes In the early days of the Baldrige, Key Themes were called Cross Cutting Themes, or Cross Cutting Issues, because they described topics that cut across the categories and items.

57. WSQA Examiner 2009 Core Values-The Foundation of the Baldrige Criteria Visionary Leadership Customer driven excellence Organizational and personal learning Valuing workforce members and partners Agility Focus on the future Managing for innovation Management by fact Societal responsibility Focus on results and creating value Systems perspective If you think of the 7 Baldrige categories as the vertical beams in a house frame, the 11 Baldrige Core Values are the cross members that provide strength to the frame.

58. WSQA Examiner 2009 Process for using Baldrige Core Values to Develop Key Themes Each Team Member: Review the applicant Read the Baldrige Core Values Write 11 post-its rating the applicant against each Baldrige Core Value with a +, 0, or – and write a few short thoughts for each core value explaining why you rated the applicant as you did. Key Themes Lead Sum the plusses and the minuses. Core Values that have many plusses, or minuses, may be candidates for Key Themes. Typically only a few will stand out. Report results to team. Draft Key themes for these high scoring values using feedback report comments and post-it thoughts as inputs.

59. WSQA Examiner 2009 Using Core Values to ID Key Themes Baldrige Core ValueScore (+,0,-) Visionary Leadership+,+,+,+,+,+ (6+) Probable KT Customer driven excellence+,+,0,0,0,- Organizational and personal learning+,+,+, 0,0,0 Valuing workforce members and partners+,+,+,+,+,- (5+) Possible KT Agility+,+,0,0,-,- Focus on the future+,+,+,0,-,- Managing for innovation0,0,-,-,-,- Management by fact-,-,-,-,-,-, (6-) Probable KT Societal responsibility+,+,+,+,+,0 (5+) Possible KT Focus on results and creating value0,-,-,-,-,- (5-) Possible KT Systems perspective+,+,0,0,-,-

60. WSQA Examiner 2009 Caveat to this approach While the Baldrige Core Values are the foundation upon which the Criteria are based, they are not examination items. Key themes are written based on comments developed relating to the criteria. The feedback report comments MUST support the Key Themes.

61. 9. Site visit preview Objectives: Understand value of site visit Set expectations

62. WSQA Examiner 2009 62 Site visit preview Site visit is awarded based on Examiners’ recommendations Scoring band (Leadership & higher is automatic) Judges’ final determination Purpose of site visit Verify strengths Clarify opportunities for improvement Improve quality of feedback report

63. WSQA Examiner 2009 63 Site visit preview Preparation Site Visit JIT training Planning calls, team assignments Develop Site Visit Issue worksheets Request documents Lead drafts interview schedule as possible

64. WSQA Examiner 2009 64 Site visit preview Day 1: Sunday (planning day) Applicant provides requested documents, equipment (printer, shredder, etc.) Draft and/or update interview schedules Review and update SVI worksheets Develop walk-around questions

65. WSQA Examiner 2009 65 Site visit preview Monday – Wednesday ON-SITE Introductory meeting (meet & greet only) Interviews – individual, teams, groups Team caucuses, lunches Walk-around questions Visit all possible sites and shifts Back at the hotel… Work on SVI worksheets, close out as possible Team reviews: “Walk-the-wall” All issues closed by end of day Wednesday

66. WSQA Examiner 2009 66 Site visit preview Thursday – Friday Draft comments based on updated SVI worksheets Review and finalize comments Review and finalize Key Themes Determine final scoring recommendation Assemble final scorebook Celebrate! Saturday: return home!

67. 10. Final scorebook preparation Objective: understand preparation of WSQA final product

68. 10. Final scorebook preparation Topics: 1.What does the team do? 2.What do the Team Lead, Editor, and Judges do? 3.What does WSQA do? 4.Final scorebook preview

69. WSQA Examiner 2009 69 What does the team do? Everything possible to save your Editor work! Follow the comment guidelines (refer to examples) Update comments during or immediately after consensus Let your Team Lead and Editor know when your comments have been updated Continue to use “applicant” throughout, not the applicant’s actual name

70. WSQA Examiner 2009 70 What does the Editor do? Attends Senior Examiner training Corrects/edits the scorebook for consistency with comment guidelines See workbook for details

71. WSQA Examiner 2009 71 What do the Team Lead, PEPD Mentor, and Judges do? Lead and Mentor prepare for presentation of final scorebook to Judges Lead and Mentor present team’s scorebook at Judges’ review Summary/highlights of comments Summary of Key Themes (Full) Scoring recommendation Judges determine award level based on team’s recommendation

72. WSQA Examiner 2009 72 What does WSQA do? Copies team scorebook into Feedback Report template Replaces “applicant” with applicant’s actual name throughout Sends Feedback Report to applicant Coordinates Executive review meeting date and participants

73. WSQA Examiner 2009 73 Final scorebook preview (Open scorebook file) Case Study final scorebook is online

74. 11. “Lite” vs. “Full” Objective: Understand distinctions in Application Process Examination Process

75. WSQA Examiner 2009 75 Lite vs. Full application process TopicLite (Assessment)Full Examination Application DeadlinesQuarterlyOnce annually Intent to Apply/ Application Form Required. Accepted throughout the year. Intent to Apply forms must be submitted at least 4 months prior to the submission of your Lite Application. Required. See Web site for due date. CriteriaLite (Assessment) Criteria, A subset of questions from the Full criteria Full version Baldrige criteria VersionCurrent or previous yearCurrent year only

76. WSQA Examiner 2009 76 Lite vs. full application process TopicLite (Assessment)Full Examination Organizational ProfileRequired, up to 5 pages Page limit, application body (not including profile, org chart, glossary) 2007: 152008: 2050 pages Examiner participationAt least one required RecognitionNoneAs determined by Judges Executive briefingYes

77. WSQA Examiner 2009 77 Lite vs. full: examination process TopicLite (Assessment)Full Examination Team size2-3 plus PEPD Mentor5-10 plus PEPD Mentor Stage 1: Independent review (Same) Stage 2: Consensus Omits Key ThemesIncludes Key Themes Judges Panel review NoneRecommends site visit OR determines award level Stage 3: Site visitNoneAs determined by Judges Post-site Judges panel review NoneDetermines final scoring and award level Feedback reportFinal report prepared by WSQA

78. TIME TABLE Always refer to WSQA website Examiner Schedule for Both Full and Lite Schedules http://www.wsqa.net/extimetable.php http://www.wsqa.net/extimetable.php Applicant Schedule http://www.wsqa.net/timeTable.php

79. WSQA Examiner 2009 79 Schedule & Time Commitment ActivityFullLite Training20 hours Individual review20-4016 Consensus preparation: comment writing8-164 Participate in consensus meeting(s)84 Add if Team Lead, Back-up, Editor16-244-8 Site visit including preparation & training60+None Total:Many!(not as many)

80. WSQA Examiner 2009 80 Discussion question Considering “Full” vs. “Lite” criteria, what are the trade-offs for each From the Applicant’s point of view? From the Examiner’s point of view?

81. 12. Summary Objectives: Summarize what we’ve learned Answer any remaining questions

82. WSQA Examiner 2009 82 High-level objectives Learn to evaluate a WSQA application through all stages of the evaluation process Do this with confidence in yourself and in the assessment process

83. WSQA Examiner 2009 83 What we’ve covered: Session 1 1.WSQA organization overview 2.Baldrige criteria overview 3.WSQA Assessment process 4.Independent review, step-by-step Preparing Key Factors Process review Results review

84. WSQA Examiner 2009 84 What we’ve covered: Session 2 5.Ethics, code of conduct 6.Consensus, step-by-step 7.Key Themes 8.Online scorebook preview 9.Site visit preview 10.Final scorebook preparation 11.Executive briefing 12.“Lite” vs. “Full”

85. WSQA Examiner 2009 85 Managing your time Put aside time in your calendar (refer to time commitments) Work with your lead & team Set meetings early Coordinate Communicate Stories to tell?

86. WSQA Examiner 2009 86 Managing your time Schedule work hours Break tasks into little pieces Reward yourself for wins Move quickly past setbacks Call your lead Call your back-up

87. WSQA Examiner 2009 87 Your questions?

88. WSQA Examiner 2009 88 Your feedback, please! Please complete surveys before leaving

89. WSQA Examiner 2009 89 THANK YOU!! Your support and participation as Examiners helps us all by helping WSQA fulfill its mission!