1. Evaluation: Prospects & Constraints
Otojit Kshetrimayum
V.V. Giri National Labour Institute, Noida

2. Impact of Training
Complex and inherently subjective nature of the judicial work
no single performance indicator can accurately measure the changes in the judge trainee’s knowledge, skills, attitudes, overall effectiveness and competence
Relying on both objective & subjective indicators

3. Impact of Training Flexibility of methodology of evaluation process
Sentencing policy and attitudes towards defendants (Bond & Lemon’s Experimental Design for Magistrates)
Impact of gender training (Questionnaires for lawyers & court staffs; interviews for judges & monitoring of judgements)
Judicial Systemic Performance model
Trial disposal rates and through put times (efficiency)
Reversal rates and appeal outcome (mistake)
Complaint rates (satisfaction)

4. Impact of Training Pragmatic choice regarding methodology
Effectiveness of training should be measured by a combination of –
Pre, post and year end focus groups/surveys;
Participant satisfaction and self-evaluation interviews
Assessment of court data & records
Personal interviews with designated officials
Independent expert appraisal

5. Constraints-1
Narrow-evaluators too often select for attention the issues that are very easy to study with available social-scientific tools.
Unrealistic- evaluators choose outcome measures that are difficult to budge.
Irrelevant- much of the data collected and analysed by evaluators is unresponsive to the real needs of the trainees.
Unfair-rarely consider the needs and wants of the trainees
Unused-evaluation reports are hardly used, seldom influence decisions about the course of the programme.
Source-Weiss

6. Constraints-2
Fears- real or imagined of having one’s mistakes exposed; of seeing one’s authority undermined; of being too complicated
Outward looking- for proving that what is done is right way of doing rather than Inward looking- for improvement of what is happening is right or wrong
Limited resources: in terms of time, money, expertise

7. Constraints-3
Selection of predominantly formative, reaction based assessment methodologies
usually, participants get a form at the end of a course or educational event, asking them to tick off little boxes, often on a five point scale, to solicit their immediate reaction to a course or event. Often, there is also a place in where participants can write specific comments. The form usually includes questions about what other courses and topics the learner would find useful.
Reliance solely on subjective qualitative data rather than to integrate objective quantitative data
Concentration on educational process rather than outcomes
General avoidance of any meaningful measurement of results in terms of enhanced judicial performance

8. Thank You