The L&D Portfolio Evaluation Model: Portfolio & Measurement Plan Decision Guide CEB Metrics That Matter™

Tips for Using This Decision Guide Portfolio Decision Guide When selecting the macro Business Outcome, use the Business Outcomes mapping page to help align to the macro outcome that makes the most sense When considering if a course is designed to provide your company with a competitive advantage, don’t overthink it; generally, your highest-investment courses will fall into that category Measurement Plan Decision Guide Keep it simple and start with 1 Measurement Plan per Portfolio If you do this, the Measurement Plan Decision Guide won’t be necessary! If you must differentiate further, establish clear criteria for “why” some courses might need a different Measurement Plan and how Course Owners will know that they are making the right choice. Update the slide accordingly. Examples: Subject matter (e.g. Leadership courses use a different Measurement Plan) Duration (e.g. Content under 1 hour uses a different Measurement Plan)

Mapping Survey Business Outcomes to Macro Outcomes The below may be helpful to consider when using the Portfolio Assignment Decision Guide: Business Outcomes which ultimately increase Operational Effectiveness: Business Outcomes which ultimately Drive Growth: Compliance Cost management Employee Retention Innovation (if focused on increasing efficiency) Employee engagement Operational Efficiency Quality Risk Safety Customer Satisfaction Sales/Revenue Innovation (if focused on new product development) Building internal business relationships Productivity Employee Engagement Note: Some outcomes could contribute to both Operational Effectiveness (bottom-line) or Driving Growth (top-line) outcomes; consider the objectives of the training and which macro outcome learners’ application of the training will contribute to most.

Portfolio Assignment Decision Guide 1 What outcome will this training ultimately impact the most? Compliance, Cost management, Employee Retention, Innovation (if focused on increasing efficiency), Employee engagement, Operational Efficiency, Quality, Risk, Safety Customer Satisfaction, Sales/Revenue, Innovation (if focused on new product development), Building internal business relationships, Productivity, Employee Engagement 2 Is the content in this course specific to our company’s processes, or expected to provide a clear advantage over our competitors? 2 Is the content in this course specific to our company’s processes, or expected to provide a clear advantage over our competitors? YES NO NO YES Let’s talk about how courses get assigned to Measurement Plans to drive data collection and are tagged to one of the 4 Portfolios for reporting. This can be accomplished using 3 questions, which are designed to help shift our thinking away from the detailed course objectives or the curriculum where the course is cataloged and toward the bigger picture of exactly how and how much value the course is expected to provide to the business. We first start by determining whether, at a high level, the course will drive bottom-line efficiency and profitability, or will drive the acquisition of new revenue. This requires some awareness of the business outcomes that contribute to each of these 2 areas, and can be provided as additional detail to those who will be using this decision tree. For example, examples of business outcomes that contribute to driving revenue growth are increasing sales, increasing customer satisfaction, new product innovation, so if a course is designed to help improve one of these, Drive Growth is the answer to question 1. If not, the course is probably more aligned to Operational effectiveness. Question 2 is designed to uncover how specific or unique this course’s content is to your company; the more tailored it is to your proprietary processes, the more we assume that the course is designed to give the company an edge over competitors. The more unique a course’s content is to your company, the more likely it was developed in-house, which also tends to be more expensive than purchasing generic content off the shelf. If the content is specific or expected to drive competitive advantage, the course belongs in whatever category you identified for question 1 and you are done. If the course is not tailored or unique to your company – maybe it’s part of an off the shelf content catalog or it’s regulatory training that everyone in your industry delivers to their employees, you should go to question 3. Question 3 helps you differentiate between whether a course belongs in the mitigate risk portfolio or the foundational skills portfolio. If the reason employees take the course is that it’s designed to prevent them from exposing the company to financial or reputational risk, like compliance with financial regulations, trade policies, or ensuring compliance with workplace safety policies, the course belongs in the Mitigate Risk portfolio. If not, the course should be assigned to the Foundational Skills portfolio. I do want to note that by going through this model, you may find that there are sales training courses that aren’t assigned to Drive Growth because the content is not tailored or proprietary, or there may be front-line manager training that is in the Mitigate Risk portfolio because the topic is interviewing without violating EEOC laws. I point this out to emphasize that the lens we are using here is the type of value the course is most aligned to, NOT the topic area. Grouping our courses topically by curricula is helpful for us to manage and for learners to find the training they need, and we can still group courses by curricula in MTM to report on curriculum-level performance, but that view is more valuable when used internal to L&D, not as a means to communicate the value that the training has delivered to the business. When an organization is configured to use the 4 core MTM Measurement Plans, the Portfolio assignment = the Measurement Plan assignment so no additional decision criteria is needed. IF there are multiple Measurement Plans per Portfolio, the next page shows how the decision guide can be extended so that after the Portfolio is determined, the correct Measurement Plan can be selected based on additional criteria. Operational Efficiency Portfolio 3 Is this course designed to shield the company from financial risk due to non-compliance with policies, laws, etc.? Drive Growth Portfolio YES NO Mitigate Risk Portfolio Foundational Skills Portfolio

Measurement Plan Assignment Decision Guide Which Portfolio is the course aligned to? Operational Efficiency Portfolio Are there any additional Measurement Plans configured in this Portfolio? Operational Efficiency Core Measurement Plan Alternate OE Measurement Plan NO YES* Does the course meet the criteria for the Core Measurement Plan? YES Mitigate Risk Portfolio Are there any additional Measurement Plans configured in this Portfolio? Mitigate Risk Core Measurement Plan Alternate MR Measurement Plan NO YES* Does the course meet the criteria for the Core Measurement Plan? YES Foundational Skills Portfolio Are there any additional Measurement Plans configured in this Portfolio? Foundational Skills Core Measurement Plan Alternate FS Measurement Plan NO YES* Does the course meet the criteria for the Core Measurement Plan? YES Drive Growth Portfolio Are there any additional Measurement Plans configured in this Portfolio? Drive Growth Core Measurement Plan Alternate DG Measurement Plan NO YES* Does the course meet the criteria for the Core Measurement Plan? YES This slide is an example of how the decision guide may be expanded for use when more than one Measurement Plan exists per Portfolio. The criteria for alternate Measurement Plans may be related to class duration, specific subject matter (e.g. Leadership or Sales courses with different outcome measures on the surveys), or different reporting needs. *configuration of additional Measurement Plans beyond the Core plans is optional

Reference: MTM Core Portfolio Measurement Plan KPI Mix Run the Business (Maintenance) Advance the Strategy (Organization Growth) Differentiating Knowledge and Skills (Drive Competitive Advantage) Generic Knowledge and Skills (Strong Foundation) The purpose of this slide is to give you a visual of the number and types of KPIs collected within each Portfolio. We can go through these in detail later, and as you can see, the courses in the upper quadrants, which are designed to drive competitive advantage by increasing operational efficiency and driving growth, will have more KPIs collected because there needs to be a greater focus on application, impact, and business outcomes. The courses in the lower quadrants, which are designed to help the business avoid cost through mitigating risk and maintaining foundational skills, will have fewer KPIs collected. You can also see that there are fewer KPIs collected for web-based training, as that modality tends to be less expensive, shorter in duration, and have less expectation of critical impact. Using a varied yet consistent and thoughtful approach like this helps to manage survey fatigue, because by using shorter, more targeted measurement plans for some course types, you can still collect the comprehensive data you need to effectively manage your more strategic, value-adding programs.