1. The Predictive Power of Self- Evaluation in Assessing Educational and Academic Outcomes Phillip J. Decker, University of Houston-Clear Lake* Roger Durand, University of Houston-Clear Lake* Lee Revere, The University of Texas Health Science Center in Houston *Co-presenters

2. Abstract  In this poster, the results of research on the power of self-evaluations of competencies to predict subsequent actual and perceived academic outcomes, especially job progression, will be reported. Background: Academic accrediting organizations are now requiring educational institutions to report student competency and alumni job outcomes. Self-evaluations are a tool increasingly being utilized to meet this requirement. The evaluation problem: Do self-evaluations predict well subsequent outcomes, including alumni job progression? Data and Methods: Survey evidence about job progression was gathered from 150 former students enrolled in a single graduate program and matched to their archived self-assessments. Analysis: Linear and non- linear predictive, statistical models with numerous controls for potentially extraneous variables were utilized in data analysis. Findings: The findings establish the predictive power of self- evaluations.

3. Background  Self-evaluations are widely used in education, performance appraisal, training programs, and career guidance among other areas.  Much of the existing evaluation literature has focused on the validity of grades when self-assigned by students; on the impact of self-assessments on knowledge improvement; and on the use of learning contracts.  Yet, there remain a number of gaps in the literature. Among these gaps are questions about how well self- evaluations predict subsequent behavior change, what exact types of subsequent behavior, and conditions of implementing self-evaluation instruments.

4. Evaluation Questions  In this evaluation research, two questions were investigated:  1. Does completing a self-assessment of competencies and/or writing an action plan increase student’s perceptions of the worth of these assignment to career progression?  2. Does successfully completing self-assessments related of competencies in combination with writing an action plan result in increased job progression?

5. IMPORTANCE OF THIS EVALUATION  We expect that this study will improve "best practices" in self-evaluation as well as enhance understanding of the predictive power of self-evaluations for evaluators in general.  Additionally, we expect that this poster will increase awareness among evaluators working in academic accreditation of the problems and prospects of utilizing self-assessments in reviewing educational programs.

6. Data and Methods  The evaluation was designed collaboratively with student and alumni participants working with professional evaluators to develop the instruments used in the study  Multi-level (alumni and student), analysis of variance and covariance evaluation designs were utilized.  Baseline survey evidence was gathered from alumni (from a ten year period) and current students.  Extensive demographic data were collected to enable determination of co-variation of such things as management experience when entering school, type of undergraduate degree, and previous self-assessment.  A measurement of self-efficacy was used to determine if self-efficacy was a moderator in the relationship between self-assessment and perceived usefulness and potential actual usefulness.

7. Process Evaluation Findings: Program Implementation  Self-Assessment, self-reflection and action plans rated very high in usefulness to career progression  Creating a portfolio rated very high in usefulness  Those who took the self-assessments seriously and rated their usefulness high progressed further in job title than does who did not  Students and alumni rated a measure of their self- handicapping behavior as useful

8. Findings: Educational Outcome Impact  Self-assessment was well received by students  Self-assessment was well received by alumni  Development of portfolios wasmoderately well received by students  Development of portfolios was well received by alumni  Assessment of Self-handicapping tendencies was well received by students  Assessment of Self-handicapping tendencies was well received by alumni

9. Findings: Academic Outcome Impact  Alumni believe self-assessment in school helped their careers  Alumni believe self-reflection papers helped their careers  Alumni believe assessment of self-handicapping tendencies helped their careers  Alumni believe portfolio construction and use helped their careers  There was a positive relationship between these beliefs and actual job title progression

10. Discussion  The results of this study will improve "best practices" in self-evaluation as well as enhance understanding of the predictive power of self-evaluations for evaluators in general.  We expect that this poster will increase awareness among evaluators working in academic accreditation of the problems and prospects of utilizing self-assessments in reviewing educational programs.  The study will help the investigator(s) examine the usefulness of self-assessment used in classes.

11. Useful, important references  Andrade, H. and A. Valtcheva, “Promoting Learning and Achievement Through Self Assessment,” Taylor and Francis Online, October 2009, pp. 12-19  Sitzman T and others, "Self-Assessment of Knowledge: A Cognitive Learning or Affective Measure?,"Academy of Management Learning & Education, Vol. 9, No. 2, 2010,169–191  Mok, MM and others, "Self-assessment in higher education:Experience in using a metacognitive approach in five case studies," Assessment & Evaluation in Higher Education,Vol. 31, No. 4, August,2006, 415–433