1. 2013-2014 Assessment Report Andrews University School of Education

2. Outline of Report Enrollment Completers Candidate Assessment Program and Faculty Assessment Recommendations

3. SED Enrollment (2013-2014) Based on data provided by the Office of Institutional Research across three semesters: Summer 2013 Fall 2013 Spring 2014

4. SED Enrollment (’13-’14) LevelCandidates Undergraduate62 MAT17 MA/MS91 Graduate Certificate11 EdS26 Doctoral (EdD, PhD)197 Undeclared/Non-degree 71 TOTAL 475

5. TLC Enrollment (’13-’14) TLC ProgramCandidates Elementary Education64 Secondary Education15 C & I (MA)12 C & I (Doctoral)31 TLC TOTAL122

6. LEAD Enrollment (’13-’14) K-12 Ed LeadCandidates Graduate Certificate11 MA1 EdS3 Doctoral16 Ed Lead TOTAL31

7. LEAD Enrollment (cont’d) (’13-’14) Higher Ed AdminCandidates MA0 EdS1 Doctoral10 HEA TOTAL11

8. LEAD Enrollment (cont’d) (’13-’14) LeadershipCandidates Undergrad Certificate MA14 EdS0 Doctoral94 Leadership TOTAL108

9. GPC Enrollment (’13-’14) GPC ProgramCandidates Special Education/ Learning Dis (MS) 9 School Counseling (MA) 9 Clinical Mental Health Counseling (MA) 29 School Psychology (EdS) 22

10. GPC Enrollment (cont’d) (’13-’14) GPC ProgramCandidates Educational Psychology (MA) 17 Educational Psychology (Doctoral) 14 Counseling Psychology (Doctoral) 32 GPC TOTAL132

11. SED Completers (2013-2014) Data Include: Graduates: Summer 2013 & Spring 2014 Degrees Conferred: December 2013 Completion Rates

12. SED Completers (’13-’14) Level Completers Undergraduate11 MAT3 MA/MS17 Graduate Certificate2 EdS8 Doctoral (EdD, PhD)25 TOTAL66

13. Completers 6-Year Trend (2008-2014)

14. Completion Rates Accreditors want to know our completion rates: How long does it take students to graduate from our programs? Program completion data for last 6 years (Office of Institutional Research). 1 year = 2, 3, or 4 semesters 2 years = 5, 6, or 7 semesters

15. Completion Rates in Years (2008-2014) DegreeLevelModeMeanN Elementary EducationBach43.8954 Elementary EducationMAT65.0015 Secondary EducationBach33.605 Secondary EducationMAT3,54.0011 Curriculum & InstructMA Curriculum & InstructEdS Curriculum & InstructDoct

16. Completion Rates in Years (cont’d, 2008-2014) DegreeLevelModeMeanN Clin Ment Hlth CounsMA 22.6221 School CounselingMA22.5618 Spec Educ/Learn DisMS2,32.506 School PsychologyEdS33.2139 Educ PsychologyMA22.4522 Educ PsychologyPhD5,6,76.003 Counsel PsychologyPhD8,910.176

17. Completion Rates in Years (cont’d, 2008-2014) DegreeLevelModeMeanN Educational LeaderMA1,32.7010 Educational LeaderEdS88.001 Educational LeaderPhD33.001 Educational LeaderGCert11.001 Higher Educ AdminMA2,32.339 Higher Educ AdminPhD66.001

18. Completion Rates in Years (cont’d, 2008-2014) DegreeLevelModeMeanN LeadershipGCert11.001 LeadershipMA44.449 LeadershipDoct57.6776

19. Candidate Assessment MTTC Scores Course Grades (CAS Courses) Course Outcomes (SED Course Rubric Data) Conceptual Framework Outcomes (SED Courses)

20. MTTC Scores Michigan Test of Teacher Certification (subject area tests) Reported annually by MDE 3-year aggregated scores (2010-2013) Reporting only scores on those tests with >/= 10 test takers Comparing Andrews with all Michigan test takers

21. MTTC Content Area Scores 3-Year Aggregated (2010-2013) AndrewsMichigan N% PassN English1291.71,84988.9 ESL1181.853890.0 Lang Arts1384.62,71779.1 Spanish1090.061091.1 Social Stud1376.92,55874.3 Elem Educ3196.87,59296.0 Guid Couns1291.740486.9 ALL TESTS15490.932,82888.3

22. Course Grades College of Arts & Sciences Courses taken by Education students Using 5-point Likert scale 3 = Satisfactory (at least C+)

23. Course Grades (2013-2014) CAS Discipline N% at 3-5MeanSt Dev Biology7100%4.290.700 Communication1080%3.801.400 Economics2100%4.500.500 English2990%4.171.116 History786%4.431.400 Integrated Science6100%5.000.000 International Languages 7100%4.860.350 Mathematics1587%4.131.024 Physics1292%4.420.954 Political Science875%3.381.111 Visual Arts9100%4.670.471

24. Course Learning Outcomes Teaching, Learning, & Curriculum Initial Teacher Education Curriculum & Instruction Foundations Graduate Psychology & Counseling Leadership Educational Leadership (K-12) Higher Education Administration Leadership

25. TLC—Teacher Ed EDTERubricN% at 3-5Mean 165/ 630 Portfolio1587%4.25 165/ 630 Personal Philosophy Paper 9100%4.59 228Clinical Observation & Interaction 1394%4.59 228Clinical Observation & Interaction (Revised) 1299%4.90

26. TLC—Teacher Ed (cont’d) EDTERubricN% at 3-5Mean 408Micro-Teaching 25100%4.36 4081: Worldview 25100%4.34 4082: Hum Grow & Change 2598%4.25 4083: Grps, Leader, Change 24100%4.23 4084: Commun, Technology 2596%4.18 4085: Research & Evaluat 2584%3.95 4086: Pers & Prof Growth 2590%4.05 4087: Content Area Knowl 2481%3.85 408Port. Intro & Closing 2596%4.51

27. TLC—Teacher Ed (cont’d) EDTERubricN% at 3-5Mean 420Literacy Intervention Strategies 1396%4.73 476Final Test Technology1498%4.71 480Philosophy of Classroom Management 24100%4.87 480Classroom Management Plan 2597%4.76

28. TLC—Teacher Ed (cont’d) EDTERubricN% at 3-5Mean 488/588Summative Eval by AU Super (General) 2098%4.30 488/588Summative Eval by AU Super (English) 396%4.15 488/588Summative Eval by AU Super (Math) 2100%4.05 488/588Summative Eval by AU Super (Social St) 1100%4.44

29. TLC—Curriculum & Instruction EDCIRubricN% at 3-5Mean 547Curriculum Foundations1100%4.60 730Theoretical Framework5100%4.57 889Project Evaluation2100%5.00

30. TLC—Curriculum & Instruction (cont’d) EDCIRubricN% at 3-5Mean 695Port 1: Christ Philosoph1100%4.80 6952: Learning Theorist1100%4.25 6953: Servant Leader1100%5.00 6954: Effective Commun1100%4.50 6955: Capable Researcher1100%5.00 6956: Lifelong Learner1100%5.00 6957: Subj Matter Expert1100%5.00

31. TLC—Curriculum & Instruction (cont’d) EDCIRubricN% at 3-5Mean 895Port 1: Christ Philosoph2100%5.00 8952: Learning Theorist2100%4.50 8953: Servant Leader2100%5.00 8954: Effective Commun2100%5.00 8955: Capable Researcher2100%4.50 8956: Lifelong Learner2100%5.00 8957: Subj Matter Expert2100%5.00

32. Foundations RubricN% at 3-5Mean EDFN 500 Personal & Profession Synthesis 3391%3.88 EDRM 505 Final Article Evaluation 3092%4.37 505Lit Review Evaluatn28100%4.65 505Proposal Evaluation2992%4.21 505Resrch Design Meth3094%4.25 EDRM 636 Program Evaluation9100%5.00

33. GPC GDPCRubricN% at 3-5Mean 514Worldview Paper14100%4.34 520Critical Review of HD Theory 2191%4.37 520Contemporary Issues in the Media 20100%9.26* 525Journal Article Review8100%4.72 525PBS Website Review4100%4.69 652WISC IV Mastery1180%3.91 *This assessment used a 10-point rubric.

34. GPC (cont’d) GDPCRubricN% at 3-5Mean 654Eval of Practicum 2.11100%4.95 654Eval of Practicum 2.21100%4.89 654Eval of Practicum 2.31100%4.89 654Eval of Practicum 2.41100%4.89 654Eval of Practicum 2.51100%5.00 654Eval of Practicum 2.61100%4.88 654Eval of Practicum 2.71100%5.00 654Eval of Practicum 2.81100%5.00 654Eval of Practicum 2.91100%5.00 654Eval of Practicum 2.101100%5.00

35. GPC (cont’d) GDPCRubricN% at 3-5Mean 810 School Psych Port 2.1599%4.36 810 School Psych Port 2.2597%4.25 810 School Psych Port 2.35100%4.07 810 School Psych Port 2.45100%4.02 810 School Psych Port 2.55100%4.28 810 School Psych Port 2.65100%3.97 810 School Psych Port 2.7599%4.48 810 School Psych Port 2.85100%4.42 810 School Psych Port 2.95100%4.16 810 School Psych Port 2.105100%4.49

36. GPC (cont’d) GDPCRubricN% at 3-5Mean School Psych Portfolio Evaluation 21100%3.09* Counseling Psychology Dispositions 2199%4.14 *This assessment used a 4-point rubric.

37. LEAD—Ed Lead (K-12) EDALRubricN% at 3-5Mean 520Vision Statement683%4.00 520Final Project7100%4.89 560Case Study883%4.08 560School Board Meeting786%4.46 565Spiritual Goals7100%4.86 565Worldview Paper7100%4.60 570Case Study Evaluation696%9.63* 570Observation5100%5.00 570Profile7100%5.00 *This assessment used a 10-point rubric.

38. LEAD—Leadership PortfolioN% at 3-5Mean Philosophical Foundations17100%3.59 Ethics, Values, & Spiritual17100%3.71 Learning & Human Devel17100%4.06 Effective Communication17100%4.00 Mentor/Coach17100%4.12 Social Responsibility17100%3.76 Resource Development; Human & Financial 17100%3.82 Legal & Policy Issues17100%3.65

39. LEAD—Leadership (cont’d) PortfolioN% at 3-5Mean Organizational Behavior, Development, & Culture 17100% 3.94 Implementing Change 17100% 3.76 Evaluation & Assessment 17100% 3.59 Reading & Evaluating Research 17100% 3.94 Conduct Research 17100% 3.82 Reporting & Implementing Research 17100% 3.88 [Individual Choice] 17100% 3.88

40. LEAD—Leadership—Brazil PortfolioN% at 3-5Mean Philosophical Foundations 36100%3.00 Ethics, Values, & Spiritual 36100%3.00 Teamwork 36100%3.00 Learn, Mentor & Human Devel 36100%3.00 Intercultural Comm & Global 36100%3.00 Social Responsibility 36100%3.00 Resourse Dev; Human & Finan 36100%3.00 Organizational Devel & Chng 36100%3.00 Organizational Behavior 36100%3.00 Evaluat & Conduct Research 36100%3.00

41. Conceptual Framework Outcomes Currently unable to access data LiveText glitch Developers are working on a solution

42. Program and Faculty Assessment Senior Surveys Course Evaluations

43. Senior Surveys Program/faculty (re: “your major”) were rated similarly by Elementary and Secondary candidates: Sec = Spanish, English, Music, Art > 4.00/5.00 mean rating on 13 out of 16 indicators Differences between Elementary and Secondary on 3 indicators that scored below 4.00:

44. Senior Surveys (cont’d) Program and Faculty differences: IndicatorElemSec All Seniors Advanced course and program offerings had sufficient depth of subject matter. 3.93 (n=14) 4.50 (n=8) 4.13 Library resources were adequate for the program. 4.14 (n=14) 3.71 (n=7) 3.89 There were adequate facilities and specialized equipment (studios, computers, instruments, lab supplies, etc.). 4.50 (n=14) 3.25 (n=8) 3.81

45. Senior Surveys (cont’d) Seek, Affirm, Change: “How much has your experience at Andrews University prepared you to…” SED mean rating >/= All Seniors on 11 out of 14 indicators Again, differences between Elementary and Secondary:

46. Senior Surveys (cont’d) Seek, Affirm, Change differences: IndicatorElemSec All Seniors Affirm: Embrace a balanced lifestyle 3.57 (n=14) 4.25 (n=8) 3.73 Affirm: Engage in creative problem solving and innovation 3.71 (n=14) 4.13 (n=7) 3.93 Change: Engage in generous service to meet human needs 3.64 (n=14) 4.13 (n=8) 3.90

47. Senior Surveys (cont’d) Spiritual Commitment: SED mean rating >/= All Seniors on all 15 indicators Negligible differences between Elementary and Secondary candidates.

48. Senior Surveys (cont’d) Comments about faculty: “The advisor was extremely helpful, and the teachers were clearly looking out for the best interest of the students.” “The enthusiasm, professionalism, and commitment of the teachers of this department kept the students engaged, involved, and learning at all times.”

49. Senior Surveys (cont’d) Comments about faculty: “High expectations allowed us to reach higher standards.” “The teachers were all highly knowledgeable in the content area.” “The professors are helpful and caring.” “The content and methods are taught well.” “First Days of School Experience is a definite strength.”

50. Course Evaluations Conducted across University in every class Evaluation data are reported for all SED and disaggregated by department “The Course” section “The Instructor” section “Overall Rating” section

51. Course Evaluations All SED IndicatorsN = 962MeanSt Dev Global Index4.150.96 The Course4.240.89 The Instructor4.250.95 Overall Rating3.971.04

52. Course Evaluations—SED Highest: “The instructor was sensitive to and respectful of all people.” Mean = 4.42 Lowest: “Timely, thoughtful, and helpful feedback was provided on tests and other work.” Mean = 4.07

53. Course Evaluations TLC IndicatorsN = 285MeanSt Dev Global Index4.160.96 The Course4.210.90 The Instructor4.290.93 Overall Rating3.991.06

54. Course Evaluations—TLC Highest: “The instructor helped me to understand the course content from a Christian perspective.” Mean = 4.45 Lowest: “Timely, thoughtful, and helpful feedback was provided on tests and other work.” Mean = 4.09

55. Course Evaluations GPC IndicatorsN = 527MeanSt Dev Global Index4.170.95 The Course4.270.88 The Instructor4.260.96 Overall Rating3.991.01

56. Course Evaluations—GPC Highest: “The instructor was sensitive to and respectful of all people.” Mean = 4.42 Lowest: “The instructor helped me to understand the course content from a Christian perspective.” Mean = 3.99

57. Course Evaluations LEAD IndicatorsN = 304MeanSt Dev Global Index4.190.84 The Course4.270.76 The Instructor4.280.83 Overall Rating4.030.94

58. Course Evals—LEAD Highest: “The instructor was sensitive to and respectful of all people.” Mean = 4.50 Lowest: “Timely, thoughtful, and helpful feedback was provided on tests and other work.” Mean = 4.12

59. Course Evaluations— Comparison Across SED IndicatorsSEDTLCGPCLEAD Global Index 4.15 4.164.174.19 The Course 4.24 4.214.27 The Instructor 4.25 4.294.264.28 Overall Rating 3.97 3.99 4.03

60. Recommendations Based on the data we’ve just examined, what should we do? What changes should we consider? Let’s “close the loop.”

61. Seven Recommendations 1.We need to get access to Conceptual Framework outcome data. Continue to work with LiveText to retrieve it. 2.We should audit the adequacy of our facilities and equipment (Senior Survey). Space, computers, technology, etc.

62. Senior Surveys (cont’d) Program and Faculty differences: IndicatorElemSec All Seniors Advanced course and program offerings had sufficient depth of subject matter. 3.93 (n=14) 4.50 (n=8) 4.13 Library resources were adequate for the program. 4.14 (n=14) 3.71 (n=7) 3.89 There were adequate facilities and specialized equipment (studios, computers, instruments, lab supplies, etc.). 4.50 (n=14) 3.25 (n=8) 3.81

63. Recommendations (cont’d) 3.We should examine how we can help our students improve their experience related to (from Senior Survey): “embrace a balanced lifestyle.” “engage in creative problem solving and innovation.” “engage in generous service to meet human needs.”

64. Senior Surveys (cont’d) Seek, Affirm, Change differences: IndicatorElemSec All Seniors Affirm: Embrace a balanced lifestyle 3.57 (n=14) 4.25 (n=8) 3.73 Affirm: Engage in creative problem solving and innovation 3.71 (n=14) 4.13 (n=7) 3.93 Change: Engage in generous service to meet human needs 3.64 (n=14) 4.13 (n=8) 3.90

65. Recommendations (cont’d) 3.We should examine how we can help our students improve their experience related to (from Senior Survey): “embrace a balanced lifestyle.” “engage in creative problem solving and innovation.” “engage in generous service to meet human needs.” 4.We should examine and verify completion data. CAEP will ask for it.

66. Completion Rates in Years (2008-2014) DegreeLevelModeMeanN Elementary EducationBach43.8954 Elementary EducationMAT65.0015 Secondary EducationBach33.605 Secondary EducationMAT3,54.0011 Curriculum & InstructMA Curriculum & InstructEdS Curriculum & InstructDoct

67. Completion Rates in Years (cont’d, 2008-2014) DegreeLevelModeMeanN Clin Ment Hlth CounsMA 22.6221 School CounselingMA22.5618 Spec Educ/Learn DisMS2,32.506 School PsychologyEdS33.2139 Educ PsychologyMA22.4522 Educ PsychologyPhD5,6,76.003 Counsel PsychologyPhD8,910.176

68. Completion Rates in Years (cont’d, 2008-2014) DegreeLevelModeMeanN Educational LeaderMA1,32.7010 Educational LeaderEdS88.001 Educational LeaderPhD33.001 Educational LeaderGCert11.001 Higher Educ AdminMA2,32.339 Higher Educ AdminPhD66.001

69. Completion Rates in Years (cont’d, 2008-2014) DegreeLevelModeMeanN LeadershipGCert11.001 LeadershipMA44.449 LeadershipDoct57.6776

70. Recommendations (cont’d) 5.We must re-evaluate our assessment system and calendar to ensure compliance with University. Data in LiveText; analysis in Weave 6.We must ensure that we are getting data from all programs—for CAEP, HLC, other accreditors, and our own improvement. 7.We need to “re-invest” in LiveText. Are we using it to potential for our needs? LiveText refresher training (especially for new faculty and staff).

71. Thank You!