1. Program Evaluation GNRS 5311 Module 4
Meredith French, BSN
Debra Morris, BSN
Anne Nance, BSN
Chloé Williams, BSN

2. Objectives Define program evaluation in nursing education.
Discuss theory-driven and method-driven evaluation methods.
Understand how the complexity theory drives a program evaluation.
Describe the elements necessary to evaluate each component of the evaluation plan.
Analyze the specific criteria determined by the Commission on Collegiate Nursing Education (CCNE).

3. Program Evaluation
Required for accreditation and approval by State Board of Nursing (Pross, 2010)
Driven historically by the accreditation process (Sauter et al., 2012)
Link to and integration of accreditation programs are still in use (Kalb, 2009; Escallier & Fullerton, 2012)
Key relationships of elements can be overlooked

4. Program evaluation theories
Underlying theoretical foundation
Method-oriented versus theory-driven
Method-oriented approaches have been popular (Sauter et al., 2012)
Method-oriented ignore views and concerns of stakeholder’s (Chen, )
Theory-driven models should be considered

5. Theory-Driven Program evaluation
Tests correct program theory and successful implementation
More scientifically sound, with causal explanations and involved mechanisms (Coryn et al., 2011)
Information generated improves existing or future programs (Chen, )
Chen’s theory-driven model used for nursing programs (Sauter et al, 2012)

6. Theory-Driven Program Evaluation
Chen’s Evaluation
Evaluation-Nursing Education Programs
Normative outcome evaluation
Develop program theory
Define activities to achieve goals
Explain linkages between programs
Identify/prioritize goals & outcomes
Normative treatment evaluation
Evaluate congruency between expected and implemented treatment
Mission & goal evaluation
Develop mission, philosophy & framework
Identify program goals & outcomes
Ensure program goals & outcomes meet professional standards
Program evaluation
Evaluation of teaching effectiveness
Adapted from Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation.
In D.M. Billings & J.A. Halstead (Eds.), Teaching in nursing (p ).

7. Theory-Driven Program Evaluation
Chen’s Evaluation
Evaluation-Nursing Education Programs
Implementation Environment evaluation
Participant evaluation
Implementer dimension
Delivery mode dimension
Implementing organization dimension
Environment Evaluation
Student dimension
Faculty dimension
Delivery mode dimension including courses
Organization dimension including leadership
Adapted from Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation.
In D.M. Billings & J.A. Halstead (Eds.), Teaching in nursing (p ).

8. Theory-Driven Program Evaluation
Chen’s Evaluation
Evaluation-Nursing Education Programs
Implementation Environment evaluation
Inter-organizational relationships
Impact evaluation
Intervening mechanisms
Generalization evaluation
Environment Evaluation
Inter-organizational relationship
Outcomes assessment
Intervening mechanisms
Generalization evaluation
Adapted from Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation.
In D.M. Billings & J.A. Halstead (Eds.), Teaching in nursing (p ).

9. Complexity theory Theory-driven evaluation method Less rigid approach
Looks at the program in it’s entirety
(Frye & Hemmer, 2012)

10. complexity theory
Ever-changing knowledge and practice patterns, therefore nursing programs are complex systems
A program as a whole is greater than the sums of its parts
Allows educators to …
structure useful program evaluation that accommodate a program’s true complexity
avoid an overly narrow approach
Provides a useful perspective for choosing an evaluation theory that serves the program more effectively (Frye & Hemmer, 2012)

11. COMMISSION ON COLLEGIATE nursing education (ccne)
Specific criteria for evaluation:
Baccalaureate
Master’s
Doctor of Nursing Practice (DNP)
Post-graduate Advanced Practice Nurse
Self-study
Standards for accreditation

12. Evaluation Components
Purpose of program evaluation
Mission
Goals
Outcomes
Implementation
Commitment
Resources
Curriculum/Teaching-Learning

13. University of Texas medical branch

14. Required documentation
CCNE
Required documentation
Standards and guidelines
Official documentation
Appointment, promotion, and tenure
Unit reports
Official correspondence
Handbooks, manuals, calendars, etc.
Advertising

15. CCNE: Resources Budget/Resources Degree Requirements
Nursing personnel
Location space
Equipment/Supplies
Degree Requirements
Full-time faculty
Student-ratio
Supporting documentation
Budget
Degree/certification/experience CV
Bargaining/policies/documents

16. CCNE: Teaching-Learning
Curriculum
Objectives
Outcomes
Nursing standards/guidelines
Teaching-learning
Environment
Competencies
Required documentation
Syllabi
Didactic/clinical learning
Student performance
Evaluations
Affiliation agreements
Documents
Assignments
Formal data and licensure

17. CCNE: Assessment Assessment and achievement Quantitative/qualitative
Completion rates
Licensure/certification rates
Employment rates
Formal complaints
Required documentation
Completion rate (each degree)
Faculty outcomes
Formal complaints
Documents
Data

18. Conclusion

19. References
Chen, H. T. (2012). Theory-driven evaluation: conceptual framework, application and advancement. In R. Strobl, O. Lobermeier, & W. Heitmeyer (Eds.), Evaluation von programmen und projekten fur eine demokratische kultur (pp ). Weisbaden: Springer. doi: / _2
Commission on Collegiate Nursing Education (2013). Retrieved from accreditation/Standards-Amended-2013.pdf
Coryn, C.L.S., Noakes, L.A., Westine, C.D., & Schroter, D.C. (2011). A systematic review of theory-driven evaluation practice from 1990 to American Journal of Evaluation, 32(2), doi: /
Doll Jr, W.E. & Trueit, D. (2010). Complexity and the health care professions. J Eval Clin Pract. 16(4),
Escalier, L.A., & Fullerton, J.T. (2012). An innovation in design of a school of nursing evaluation protocol. Nurse Educator, 37(5), doi: /NNE.0b013e318262eb15
Frye, A.W., & Hemmer, P.A. (2012) Program evaluation models and related theories: AMEE guide no Medical Teacher, 34, Doi: / X
Kalb, K.A. (2009). The three Cs model: the context, content, and conduct of nursing education. Nursing Education Perspectives, 3(30),
Sauter, M.K., Gillespie, N.N., & Knepp, A. (2012). Educational program evaluation. In D.M. Billings & J.A. Halstead (Eds.), Teaching in nursing (pp ).