1. Data Analysis Retreat: May 19-21, 2010 School of Education Campbellsville University

2. Why? Standard 2: Assessment System and Unit Evaluation The unit has an assessment system that collects and analyzes data on –applicant qualifications, –candidate performance, –graduate performance and –unit operations Purpose –to evaluate and improve the performance of candidates, the unit and its programs.

3. Why? NCATE Standard 2b: Candidate assessment data are regularly and systematically collected, compiled, aggregated, summarized and analyzed to –Improve candidate performance –Improve program quality –Improve unit operations

4. Why? NCATE Standard 2c The unit regularly and systematically uses data, including candidate and graduate performance information to evaluate the efficacy of its –Courses –Programs –Clinical experiences

5. Why? NCATE Standard 2c The unit analyzes program evaluation and performance data to initiate changes in –Programs –Unit operations

6. Overview Data for 2009 –Compared to 2006-2008 Undergraduate Programs –CAPs 1, 2, 3, 4 –Praxis II –Diversity Proficiencies Graduate Programs: MAE/SED/Rank I –CAPs 5, 6 and 7 Faculty/Course Evaluations

7. CAP 1: Admission to TE 2009 Number Admitted Sp09: 27 F09: 22 2009 Total: 49 Analysis: 9-14 < previous three years GPA Sp09 3.46 F09 3.44 09 Av. 3.45 06-08 Av. 3.40 > previous three years

8. CAP 1 Competency Exams: 2009 ACTPPST: RPPST: MPPST: W SP09 23.60177.20174.60175.60 F09 23.28176.25172.25173.75 09 Av. 23.45176.78173.56174.78 06-09 Av. 23.51177.21174.53174.58 Analysis < 07, 08 <06 Overall average >06 < 07, 08 <overall average >previous 3 years and overall average < 06, 07 >08 & overall average

9. CAP 1 Dispositions 2009 Average 2009 4.36 Average 2006-2009 4.17 Sp 09 4.35 ; F 09 4.37 Highest (4) Honest/Ethical 4.53 (1) Caring 4.40 Lowest (3) Values Ind. Differences 4.27 (5) Responsible Class. Behavior 4.27

10. CAP 1 Dispositions 2009 SP/Fall2009 Average Analysis Overall4.35, 4.374.36< all previous 1-Caring 4.48, 4.374.40< all previous 2-Respects 4.35, 4.394.37<all previous 3- Values individual Differences* 4.25, 4.284.27< all previous 4-Honest & ethical 4.56, 4.524.53<all previous 5.Responsible Classroom Behavior* 4.21, 4.294.27<all previous

11. CAP 1 Interviews 2009 Overall Interview GPA Disposi- tions Experience Oral Comm. *Impact/ Student Learning Sp094.003.474.273.604.203.53 F094.563.704.404.304.204.30 09 Av.4.223.564.323.884.203.84 06-09 Av. 3.963.434.043.534.113.55 Analysis > Previous years > Previous years > Previous years F09 > Previous years

12. CAP 1: Admission by Major 2009 Major# Adm.09Analysis P-5 Sp=7; F=4 (11) < last three yearsF09 ‘way’ down 5-9 Opt.1 Eng 3 Math 3 Sc 1; SocSt 1 F09=8 <previous 3 years 2 in fall 09 1 English; 1 Math. 5-9 Opt. 2 006=8 07=4 08=2 8-12 Eng=1/2 Math=3/1 Bio=1/0 Chem=0/1 SS=2/5 E=3,1, 6 M=3, 4, 4 B= 0, 1, 2 SS= 6, 7, 3 1 chemistry last four years; biology & chemistry lowest P-12 Art=0/2 Music=4/3 PE=1/1 Health=2/1 Art=2, 3, 1 Music=10, 5, 10 PE=5, 5, 5 Health=0, 0, 0 Art/ Physical Education- lowest for admissions

13. CAP 1: Diversity 2009 Age: most common range—16-25, 83/91% Gender 76% females ; 24% males Ethnicity 96/91% white; Black 4%; Hispanic 2% Exceptionality 26% < previous 21% GT;.5% LD; 1% visual;.5 ADD; 76% unidentified 2 nd Language 15% 7% Spanish Religion Baptist- 59%; Christian-26%; Catholic-7%; Methodist-4%

14. CAP 1: Diversity 2009 Live at home: 13%, < all previous Work: 30%, < all previous Income: 80% Blank 2% 25,401-42,802 9% 42,803-67,326 More did not answer. Father High school: 65%; College: 11%; Graduate: 11% Mother: High school: 54 %; College: 22%; Graduate: 13%

15. CAP 2: Continuation in TE 2009 39 Approved –Overall GPA: 3.40/3.29 3.36 average < all previous and 06-09 average –Professional GPA: 3.75/3.70 3.73 average < all previous –Field Hours: 40, > previous –PPD Hours: 11, > previous

16. CAP 2 Dispositions 2009 SP/Fall2009 Average Analysis 1-Caring 4.52, 4.134.43< 07 2-Respects 4.54, 4.184.45<08 3- Values individual Differences* 4.49, 4.184.42< all previous 4-Honest & ethical 4.66, 4.354.59>all previous 5.Responsible Classroom Behavior* 4.43, 4.354.41<08

17. CAP 3: 2009 Admission to Student Teaching Total 2009: 47 GPA: –Overall GPA: 3.39 –Major GPA: 3.42 –Prof. GPA: 3.77 Dispositions: –Overall: 4.59 –Education Professor: 4.24 –Cooperating Teacher: 4.77 –Self Evaluation: 4.53 –Instrument changed fall, 2006

18. CAP 3: Field/PPD Hours 2009 Total: Field hours-130.35; PPD-32.57 Type: Assisting 96%, consistently high Instructing 91%, consistently high Observing 96%, consistently high Research 55%, comparable to previous Tutoring 64%, < f08 Participating in education related community events Interacting with families of students Attending school board meetings Diversity: Ethnicity 92%, comparable to 06 & 08 Exceptionality 89%, < previous years Gender 89% Linguistic * 55%, < previous Race 81%, < 07, 08 Religious 62% Socio-econ. 87%

20. CAP 3: Portfolios 2009 Portfolio Mean Holistic Score 2.49 < fall, 2008 2.70 Changeover to KTS in fall 2008 2009 Highlights –Highest KTS 3 Climate 2.90 KTS 1 Content 2.78 –Lowest KTS 5 Assessment 2.44 KTS 7 Reflection 2.46 KTS 8 Collaboration 2.54

21. CAP 3 Portfolios Holistic Scores: Past NTS-Spring, 2008 Highest –NTS 4 Assessment 2.73 –NTS 2 Climate 2.68 Lowest –NTS 5 Reflection 2.54 –NTS 6 Collaboration 2.46 –NTS 3 Implement 2.42 KTS Fall, 2008 H ighest –KTS 3 Climate 2.91 –KTS 10 Leadership 2.85 Lowest –KTS 5 Assessment 2.51

26. CAP 3: Portfolios by Majors- 2009 Comparisons: –P-5: Highest KTS 3 Climate; Lowest KTS 4 Design –5-9 Highest KTS 3 Climate; Lowest KTS 6 Technology, KTS 5 Assessment –8-12: Highest KTS 3 Climate; Lowest KTS 5 Assessment KTS 7 Reflection –P-12: Highest KTS 3 Climate; Lowest KTS 8 Collaboration KTS 5 Assessment

28. CAP 3 Growth Areas 2009 KTS 5 Assessment –Communicating learning results 2.14 –Allows student self assessment 2.20 KTS 7 Reflection –Uses data to reflect and identify areas for professional growth 2.26

29. CAP 4: 2009 63 Approved Spring/Fall Represents a steady increase over last four years Cumulative GPA: 3.55 > than previous Major GPA: 3.62 (3.70/3.49) Overall > than last four years; f09 is <last two years Professional GPA: 3.76 Comparable to previous

30. CAP 4 Clinical Experiences 2009 Diversity –Ethnicity97% >08, 07 –Exceptionality95% –Language67% (59/81%) < –Race92% < 08, 07 –Religion: 76% > –Socioeconomic: 98% > –Gender:98% > than previous

31. CAP 4 Clinical Experiences 2009 Assisting: 100% > last 2 Instructing: 100% = to previous Observing: 98% > 08, 06 Research: 75% < 06, 07 Tutoring: 79% < previous Attend school board meetings Interacting with families Participate in education related community events

32. CAP 4 Dispositions 2009 * F06: New Instrument Analytic SP/F Analysis Overall4.82/4.64> 06,07; < 08 1. Caring4.81/4.62> 06,07; < 08 2. Respects4.88/4.70> 06, 07; < 08 3. Values ind. Diff.* 4.72/4.54> 06, 07; < 08 4. Honest & ethical 4.93/4.74> 06; < 08, =07 5. Responsible*4.75/4.57> 06, 07; < 08

33. CAP 4: Dispositions 2009 Holistic Analysis Cooperating Teacher 1 4.81> 06, 07; < 08 Cooperating Teacher 2 4.73> 06, < 08, 07 Supervising Teacher 4.63> 06, < 08 Self Evaluation4.85> 08, 07, 06

34. CAP 4 Dispositions 2009 Univ. Supv. Self #1 Caring4.624.87 #2 Respects others 4.634.90 #3 Values indiv. Diff.*4.524.76 #5 Honest/ethical4.784.92 #6 Responsible 4.574.77 classroom behavior*

35. CAP 4 Portfolios Fall 2009 Holistic: 2.90/2.79 Analysis KTS 12.92/2.77 2 nd Highest (Content) KTS 22.81/2.76 KTS 33.00/2.92 Highest (Climate) KTS 42.89/2.78 KTS 5*2.80/2.69 2 nd Lowest (Assess.) KTS 62.81/2.54 (Lowest NTS) KTS 7*2.79/2.80 Lowest (Reflection) KTS 82.84/2.85 KTS 92.91/2.92 (Highest NTS) KTS 102.91/2.81

36. CAP 4: Portfolios by Major, 2009 Portfolio Scores: KTS P-5 2.95 < 5-9 3.00 8-12 –Biology –English2.33< –Math3.00 –Soc. St.2.75 < P-12 –Art3.00 –Music2.82< –PE 3.00

37. CAP 4 Portfolio Growth Areas 2009 KTS 5 –Lowest is ‘Communicates learning results’ –Next lowest are ‘Describes, analyzes and evaluates performance data’ & ‘Allows student self assessment’ KTS 7 –Lowest: ‘Uses data to reflect on and identify areas for professional growth’

38. CAP 4: Univ. Supv./Coop. Tchr, Eval. 1-3 2008 NTS/KTSUniv. SCoop.TAnalysis KTS 1 Content2.55-2.942.64/2.96 CT-Improved from MT to final KTS 2 Design2.61-2.952.71/3.00 KTS 3 Climate2.72-2.922.79/3.00 KTS 4 Implement 2.44-2.882.59/2.98 CT-Improved most KTS 5 Assessment 2.33-2.852.58/2.90 CT-Began lowest KTS 6 Technology 2.69-2.892.77/2.98 KTS 7 Reflection 2.40-2.952.66/2.91 KTS 8 Collaboration 2.33-2.982.79/2.93 US-3 rd most improved KTS 9 Professional Gr 2.21-3.002.58/2.95 US-began lowest US-most improved KTS 10 Leadership _____________ Holistic A + B 2.30-2.982.72.2.92 ________ 2.56/2.96 US-2 nd most improved

39. CAP 4: Video Evaluations-Prof./Self KTS Professor Self Evaluation Analysis KTS 1 Content 2.89/2.902.87/3.00 KTS 2 Design 2.94/2.672.89/3.00 KTS 3 Climate 2.91/2.852.92/3.00 KTS 4 Implement 2.82/2.862.68/3.00Self-lowest overall Professor-2 nd lowest overall KTS 5 Assess 2.91/2.722.81/3.00Self-2 nd lowest overall Professor-lowest overall KTS 6 Technology 3.00/2.812.88/3.00

40. CAP 4: Interview Ratings 2009 IndicatorRatingTrend over semesters Poised 4.79 > Last 2 years, highest Articulate 4.56 < Last 2 years Content Knowledge 4.54 < than 2/3 Pedagogy 4.49 > 2/3 years, lowest over time Reflection 4.59 > Last 3 years Q1: Meet KTS 4.54 > Last 2 years Q2: Student Learning Overall 4.45 4.56  Previous years

41. CAP 4: Praxis Data--2009 PLT: Passing Score of 161 –P-5: 0522 177.58 < 06, 08 –5-9: 0523 English-168; Math-180; Science; 176; Soc. St. 181 Soc. St/English-185; Math/Soc.St.-180 –8-12: 0524 English-158; Biology-179; Soc. St.-176; Math-184 –P-12 (Choice) Music169 Art169 PE/Health174

42. CAP 4: Praxis II Data 2009 LevelPassingMean P-5148168.42 5-9 –English 158159 –Math148166 –Science144148 –Social St.149154 –SocSt/Eng149/158160/192 –Math/SocSt148/149166/168

43. Praxis II Data 2009 PassingMean Biology146185 English160/155181/158 Math125/141139/161 SocSt.151/159163/165 Art154/158160/161 Music145/154148/161 PE/Hlth147/151/630152/158/677

44. Diversity Proficiencies: NTS Sp, 2008 NTSCAP3 Port. CAP4 Port.* NTSCAP4 Video CAP4 Co.T CAP4 Un.T 1c2.552.601-4,54.674.55 1d2.872.854.62 1f2.652.73 2b2.742.852-34.784.40 2d2.832.85 3c2.572.493-44.64 3d2.752.60 4d2.672.584-14.61 5a2.712.835-14.73 6a2.582.708-2,54.73 8c2.542.688-1,34.474.644.31 8e2.632.739-94.584.20 9d2.542.489-83.934.70

45. Diversity Proficiencies: KTS 2008/09 KTS CAP 3 Portfolio CAP 4 Portfolio Coop. Teacher Univ. Supervisor Video Evaluation Indicator 1b 2.64/ 2.65 2.73/ 2.70 2.81*/ 2.84 2.77/ 2.88 2.76 2.72/2.78 Connects content to life experiences 1d 2.49/ 2.50 2.58*/ 2.83 2.82/ 2.83 2.79/ 2.86 2.62 2.72/2.68 Guides students to understand content from various perspectives. 2b 2.55/ 2.58 2.84/ 2.81 2.86/ 2.85 2.90/ 2.93 2.69 2.78/2.70 Uses contextual data to design instruction relevant to students. 3c 2.77/ 2.67 2.84/ 2.80 2.88/ 2.88 2.77/ 2.81 2.78 2.71/2.80 Values & supports student diversity & addresses individual needs 4b 2.43*/ 2.33* 2.78/ 2.67 2.82/ 2.80** 2.67*/ 2.85** 2.45* 2.69*/2.76 Implements instruction based on diverse student needs and …. 5c 2.73/ 2.38* 2.81/ 2.50* 2.91/ 2.85 2.80/ 2.73* 2.68 2.81/2.65* Regularly integrates technology …meet diverse student needs.

46. CAP 1 IECE: Fall 2009 Approved: 29 all locations Overall GPA: 3.37 ACT: 22.13 PPST: 176.25 Reading 172.75 Mathematics 174.50 Writing Note: IECE programs began in Fall 2009. Data for IECE are presented separately from other data; the IECE data will also be disaggregated by location.

47. CAP 2: IECE Fall, 2009 Number approved: 6 Grade point average: 3.40 Foundations courses: 3.77 Field Hours: 177.50 PPD Hours: 20.25

48. IECE: Diversity Data 2009 Age: 42% 26-35 38% 16-25% Gender: 100% F Exceptionality: 97% no 2 nd language: 100% No Father: 12% College Mother: 27% College Religion: –42% Christian –24% Baptist –15% Catholic – 9% Methodist

49. IECE Disposition Data: Fall, 2009 Prof. Prof. Prof. Self Overall Caring: 4.0 4.83 4.85 4.63 4.76 Respects: 4.0 4.50* 4.85 4.69 4.63 Values: 4.0 4.71 4.62* 4.56* 4.63 Honest: 5.0 4.92 4.85 4.88 4.89 ProfResp. 5.0 4.46 4.83 4.63 4.60*

50. MAE: CAP 5 Admission 2009 2009: 2 (13-2008) <from previous GPA: 3.69 > all previous years Dispositions Mean: 5.00 > all previous

51. MAE: CAP 6 & 7, 2009 CAP 6 Continuation –Number approved: 9 –Number of hours completed: 19 –GPA: 3.96 CAP 7 Exit –Number approved: 14 –GPA: 3.93 –Action Research project grade: 93.71/94.38< previous year

52. MAE: Diversity Data 2009 Age: 60%-26-35; others 16-25 Gender: 82% female, > previous Race/Ethnicity: 100% white Exceptionality: 0%, < previous Father’s Education: 100% HS Mother’s Education: 35% College Second Language: 0% Religion: –40% Baptist –25% Methodist –20% Christian

53. Rank I Data: 2009/08 CAP 5: –Number admitted 13/9 DOSE certification: 2/4 Gifted endorsement: 2/2 Professional specialty: 9/3 –GPA: 3.90/3.84 CAP 6: –Number approved: 7; GPA: 3.96 CAP 7: –Number approved: 7; GPA 3.95

54. Rank 1 Diversity Data 2009 Age: 50% 36-45; 44% 26-35 Gender: 99% female Race: 100% white Exceptionality: 20% (66% GT; Asthma 33%) Father: 58% hs; 31% college Mother: 66% hs; 32% college 2 nd Langauge: 7% Spanish Religion: 46% Baptist; 25% Christian; 12% Catholic

55. Rank 1 Program Data 2009 Dispositions: –CAP 5: 4.41 –CAP 6: 4.52 –CAP 7: 4.78 –Overall: 4.60 Culminating Project Average: 198/200

56. SED TRAC 2 Data 2009 CAP 1 –Admitted 06 & 07: 6 –GPA: 3.75 CAP 2 –Approved 06 & 07: 7 –GPA: 3.14 –Field Hours: 73 CAP 3 –Approved 06-08: 6 –GPA: 3.17 –Field Hours: 103.14

57. SED TRAC 2 DATA 2009 Field Experiences CAP 3 CAP 4 –Assisting: 23% 100% –Instructing: 57% 100% –Observing: 57% 50% –Research: 55% 50% –Race/Ethnicity: 100% 50% –Exceptionality: 57% 100% –Gender: 57% 50% –Language: 2% 0% –Religion: 2% 0% –Socioeconomic-- 50%

58. SED TRACs 3 & 4 Data 2009 CAP 5 –61 admitted (48-T3; 13-T4) < previous years –Undergraduate GPA T3: 2.87; T4: 2.70 < than previous –GRE Verbal: T3: 363; T4: 334 T3> than previous; T4 < previous Quantitative: T3: 375; T4: 385 < 08 Analytical: T3: 3.45; T4: 3.13 T3: >previous years: T4: < previous years

59. SED CAP 6 Tracs 3 & 4--2009 Approved 59/79 T3: 43; T4: 16 < 06, 08 GPA T3: 3.91 T4: 3.66 T3: <07, 08 T4: < previous Field Hours T3: 38 T4: 38 >06, 07 < 08

60. SED: TRACs 3 & 4, CAP 7 2009 Approved T3: 38; T4: 22 Total: 60 < previous GPA T3: 3.88; T4: 3.83Similar Field Hrs. T3: 72; T4: 45<07, 08 Praxis II 0353: 157 Pass T3: 159; T4: 167 0542: 172 Pass T3: 180; T4: 179 T3 < previous T4> 07, 08

61. SED: TRACs 3 & 4 Diversity 2009 Age56%: 26-35 years 15%: 16-25 years 22%: 36-45 years Similar to previous years and average Difference in Trac 3 & 4 Gender73%F 27% maleFewer males than previously Ethnicity85.0% White 12.5% Black 2.5% Asian Whites increasing Exceptionality0%< previous Income28%-$25-43,000 17%-$10-25,000 15%-$42-67,000 Fewer in lower two brackets Father55%-High School 19%-College Similar to previous years

62. SED TRACs 3 & 4 Diversity 2009 Mother59%- HS 8%-College 19%-Graduate > Graduate Language 4% Yes< previous Spanish Religion29%-Baptist 30%-Christian 8%-Catholic 14%-Methodist Fewer Methodists

63. SED: TRACs 3 & 4, CAP 7 Portfolios 2009 KTS 12.76 Content: > 06-08 KTS 22.76 Design: > 06-08 KTS 32.81 Climate: > 06-08 Highest KTS 42.76 Implement: > 06-08 KTS 52.62 Assess: > 06-08 KTS 62.58 Technology: > 06, 08; < 07 Lowest KTS 72.63 Reflect: > 06-08 KTS 82.70 Collaborate: > 06-08 KTS 9 KTS 10 2.70 2.73 Professional Growth: > 06, 07 Leadership: NA

64. SED TRACs 5 & 6, CAP 5 2009 ApprovedT5: 42; T6: 7 Total 49 > 06, 07; < 08 GPA3.37/3.06 3.85/4.0 T5: > previous; T6 < 08, 06 Age50%: 26-35 36%: 16-25 Fewer from 36-45 years Gender53%-Female 47%-Male Number of males increasing Ethnicity74%-White 10%-Hispanic 15%-Black > Blacks Except.0% Yes> 07, 05; GT, Hearing, Visual

65. SED: TRACs 5 & 6 Data 2009 CAP 6: # App.54Comparable to 08; > 07, 06 GPA 3.83/3.88 Field Hours33/37<07, 08 CAP 7: # App.43, 8More T5, Fewer T6 GPA3.88/3.92Comparable Field Hours68/71Comparable Praxis II 0353 T5--176.84 T6--179.67 > previous 0542 T5--184.6 T6—177.8 T5 06, 07 T6 06

66. Faculty Evaluations: Fall, 2008 All scores 4.0 & above, except 24 (Class.) Highest scoring questions –Q14: 4.55 Provides prompt feedback –Q11: 4.53 Communicates effectively… –Q16: 4.44 Available during office hours Lowest scoring questions –Q10: 4.00 Communicates interest in course… –Q1: 4.03 Objectives/requirements made clear –Q13: 4.09 Routinely holds class full period Scores lower than campus wide –Q10: -.26—Communicates interest; –Q1: -.24—Objectives requirements made clear –Q2: -.08 (Relates); Q3: -.05 (Mat.); Q17: -.05 (Compared.)

67. Faculty Evaluation Data f09 Regular Program: –Instructor give sufficient number of tests to ensure fair evaluation (4.15/4.28) –Materials are utilized (4.23/4.27) SED –I feel free to raise questions and participate (3.70) –Variety of presentation techniques (3.97) ECE –Grading practices are fair (4.60/4.28) IECE –Course is well organized (4.20/4.25)

68. Additional Data Needed Faculty Evaluations –Evidence of using data for program improvement Plan for Follow-up Surveys of recent grads from all programs SACS: employment status of graduates

69. Title II Update Westat: Federal Database Admission/completion/certified Annual Goals –Shortage areas of math, science, special education, ELL instruction 7 Assurances 1.Training responds to needs of local educational agencies 2.Training linked to needs of schools 3.Core academic subjects 4.Training for children with disabilities, 5.Limited English proficiency 6.Low income families, 7.Urban/rural schools

70. Summary: Growth Areas Traditional Programs –Dispositions: Lowest at CAP 1, 3 and 4 3. Values individual differences 5. Demonstrates responsible classroom behavior –CAP 1 & 4 Interviews: Impact on student learning (3.84/4.45) –KTS (CAP 3 & 4, Video) Assessment (communicates learning results) –CAP 3 lowest for overall,5-9, 8-12, P-12, SED Reflection (using data for pd) –CAP 3 lowest for overall 2.46 –Diversity 4b implement instruction based on individual student needs 5c regularly integrate technology to meet diverse student needs Ensure field/clinical experiences with linguistic diversity

71. Summary: MAE/Rank I Growth Areas MAE: –Data on major assessments used in the program-Action Research project –Capability to show evidence of KTS –Diversity: 100% white Rank I –Collect evidence of KTS

72. Summary: Growth Areas-SED TRACs 3 & 4: –Portfolio: KTS 5/6 lowest (assessment & technology) TRACs 5 & 6: –Major assessment data for KTS

73. Growth Areas: Progress Collaboration: improvement New Teacher Survey –Old: #2 Community/background in planning Improved; still –.12 –New: #5 Designing/Using formative assessments to provide feedback to students and guide their learning -.18 Diversity –Recruit diverse candidates and faculty Need to reinstate Pathways or other ‘good faith efforts’ –Implement unit diversity plan