CONTEXTUAL FACTORS OF EVALUATION PRACTICES Evaluation is a must in any program planning model Evaluation is a must in any program planning model There is a gap between model and practice of program evaluation There is a gap between model and practice of program evaluation In general evaluation models suggest: In general evaluation models suggest: Formal Formal Systematic Systematic Comprehensive Comprehensive

Purpose of the Study To identify how contextual factors shape evaluation practices To identify how contextual factors shape evaluation practices

Specific Research Questions 1. What are the evaluation practices of different adult and continuing education agencies in Malaysia 2. What are the contextual factors involved in the evaluation practices of the agencies

RESEARCH METHODOLOGY Qualitative multiple case study Qualitative multiple case study Purposive Sampling Purposive Sampling Three types of organization Three types of organization Panel of experts Panel of experts Four criteria Four criteria

Data Collection Data Collection Triangulation Triangulation Semi structured interview guide Semi structured interview guide Official documents Official documents Data Analysis Data Analysis Within case Within case Cross-case Cross-case

Findings Type of organization: Type of organization: Agency A – Continuing education Agency A – Continuing education Agency B – Agricultural extension Agency B – Agricultural extension Agency C – Entrepreneurial development Agency C – Entrepreneurial development

Numbers of respondents Numbers of respondents Agency A – 4 Agency A – 4 Agency B – 4 Agency B – 4 Agency C – 8 Agency C – 8 Evaluation unit Evaluation unit Agency A - Publication & Consultancy Bureau Agency A - Publication & Consultancy Bureau Agency B – Research & Evaluation Unit Agency B – Research & Evaluation Unit Agency C – Every departments conduct own evaluation activities Agency C – Every departments conduct own evaluation activities

Evaluation activity Evaluation activity Agency A – Informal process evaluation & end of course evaluation Agency A – Informal process evaluation & end of course evaluation Agency B – Periodic evaluation throughout the program & oral evaluation Agency B – Periodic evaluation throughout the program & oral evaluation Agency C – Periodic, informal evaluation throughout program and seminar Agency C – Periodic, informal evaluation throughout program and seminar

Contextual Factors 1. Importance of evaluation to the organization *Mission of public organizations *Mission of public organizations *Organizational structure of the organizations *Organizational structure of the organizations *Management’s attitude toward evaluation *Management’s attitude toward evaluation

2. Ability of the evaluators 3. Culture of the clientele 4. Tradition bound evaluation practices

Conclusions 1. Minimalist formal evaluation approaches *Reaction level *Monitor input *Findings are not fully utilized

2. Not comprehensive nor systematic 3. The four contextual factors *Interconnected *Collectively influence practice