Ardianto Prabowo Indira Dwiajeng A. Maria Angela Masruroh Susan Kuncoro

Measuring Indicators, Satisfaction, and Learning

1. Indicators2. Satisfaction3. Learning4. Application5. Business Impact6. Return on Investment7. Intangible Benefits

Typical Indicators include: The number and variety of programs The number of employees participating in a leadership development program Total number of hours of learning activity per employee Various enrollment statistics, including demographics of participants, participation rates, completion rates, etc.

Investment in leadership development programs reported in a variety of ways. (Total cost, cost per employee, direct cost per participant, and cost as a percent of payroll are common ways.) Cost recovery, if there is a charge back system. The types of delivery mechanisms.

Indicators usually show the degree of management’s commitment to leadership development and provide a brief view of the mix of programs offered

The most widely used data source for reaction and satisfaction data is the program participants

 Questionnaires  Used to obtain subjective information about participants, as well as to objectively document measurable business results for an ROI analysis.

 Surveys  represent a specific type of questionnaire with several applications for measuring training success.  Used in situations where attitudes, beliefs, and opinions are captured only, whereas a questionnaire has much more flexibility and captures data ranging from attitude to specific improvement statistics.

Surveys can have yes or no responses a range of responses (strongly disagree…strongly agree) five-point scale Questionnaires open-ended question Checklist two-way question multiple-choice ranking scale

 Determine the specific information needed  Involve management in the process  Select the type(s) of questions  Develop the questions  Check the reading level  Test the questions  Address the anonymity issue  Design for ease of tabulation and analysis  Develop the completed questionnaire and prepare a data summary

The most common types of feedback solicited:  Progress with objectives. To what degree were the objectives met?  Program content. Was the content appropriate?  Instructional materials. Were the materials useful?  Pre-work materials. Were the pre-work materials necessary? Helpful?

 Assignments. Were the out-of-class assignments helpful?  Method of delivery. Was the method of delivery appropriate for the objectives?  Instructor/facilitator. Was the facilitator effective?  New information. How much new information was included?  Motivation to learn. Were you motivated to learn this content?

 Relevance. Was the program relevant to your needs?  Importance. How important is this content to the success of  your job?  Registration/logistics. Were the scheduling and registration efficient?  Facilities. Did the facilities enhance the learning environment?

 Potential barriers. What potential barriers exist for the application of the material?  Planned improvements/use of material. How will you apply what you have learned?  Recommendations for target audiences. What is the appropriate audience for this program?  Overall evaluation. What is your overall rating of the program?

 Monitor customer satisfaction.  Identify strengths and weaknesses of the program.  Develop norms and standards.  Evaluate the leadership development staff.  Evaluate planned improvements.  Link with follow-up data.  Market future programs.

 Use a simple questionnaire  Collect data early and react quickly  Pay attention to participants