Evaluation: Prospects & Constraints Otojit Kshetrimayum V.V. Giri National Labour Institute, Noida otojit@gmail.com

Impact of Training Complex and inherently subjective nature of the judicial work no single performance indicator can accurately measure the changes in the judge trainee’s knowledge, skills, attitudes, overall effectiveness and competence Relying on both objective & subjective indicators

Impact of Training Flexibility of methodology of evaluation process Sentencing policy and attitudes towards defendants (Bond & Lemon’s Experimental Design for Magistrates) Impact of gender training (Questionnaires for lawyers & court staffs; interviews for judges & monitoring of judgements) Judicial Systemic Performance model Trial disposal rates and through put times (efficiency) Reversal rates and appeal outcome (mistake) Complaint rates (satisfaction)

Impact of Training Pragmatic choice regarding methodology Effectiveness of training should be measured by a combination of – Pre, post and year end focus groups/surveys; Participant satisfaction and self-evaluation interviews Assessment of court data & records Personal interviews with designated officials Independent expert appraisal

Constraints-1 Narrow-evaluators too often select for attention the issues that are very easy to study with available social-scientific tools. Unrealistic- evaluators choose outcome measures that are difficult to budge. Irrelevant- much of the data collected and analysed by evaluators is unresponsive to the real needs of the trainees. Unfair-rarely consider the needs and wants of the trainees Unused-evaluation reports are hardly used, seldom influence decisions about the course of the programme. Source-Weiss

Constraints-2 Fears- real or imagined of having one’s mistakes exposed; of seeing one’s authority undermined; of being too complicated Outward looking- for proving that what is done is right way of doing rather than Inward looking- for improvement of what is happening is right or wrong Limited resources: in terms of time, money, expertise

Constraints-3 Selection of predominantly formative, reaction based assessment methodologies usually, participants get a form at the end of a course or educational event, asking them to tick off little boxes, often on a five point scale, to solicit their immediate reaction to a course or event. Often, there is also a place in where participants can write specific comments. The form usually includes questions about what other courses and topics the learner would find useful. Reliance solely on subjective qualitative data rather than to integrate objective quantitative data Concentration on educational process rather than outcomes General avoidance of any meaningful measurement of results in terms of enhanced judicial performance

Thank You