University of Arkansas Faculty Senate Task Force on Grades Preliminary Report April 19, 2005

Mandates for the Task Force Does grade inflation at the U of A exist and is it a significant issue of faculty concern that may require policy change? Are grades and any perceived or real grade inflation with time, or uneven grade distribution linked to uniform faculty evaluations (Purdue System)? If there IS a relationship between Teaching Evaluations and Grading, … is it a significant problem that should be addressed? What are Task Force Recommendations related to grades, grade definitions, etc., … that should be a faculty matter.

U of A not Unique! Academe Interested in Grades and Grading Practices Duke University Harvard Dartmouth Southwest Missouri State University University of North Carolina Princeton

Assignment of Grades Philosophical Discussion (Faculty) – Why do we assign grades? – What does a grade represent? – Types Criterion or Norm Referenced Pragmatic Examination (Task Force) – What grades were assigned? – Are there patterns in the assignment of these grades that may represent a greater systemic problem?

Student Evaluation of Instruction Philosophical Issues (Faculty) – Purpose of student evaluations? – Does the Purdue System provide the necessary information to evaluate instruction? – Differentiating a high rating from effective instruction and possible “inflation” of grades Pragmatic Issues (Task Force) – Are There Patterns in Student Evaluations Associated with Overall Grades Assigned Type of Course Faculty Rank

Increasing Grades? Since mid-60s universities have seen undergraduate GPAs steadily increase College Remedial Courses – Annual rate approaches 60% in Arkansas – 34% of Students in Arkansas Identified as “college ready” Numerous theories exist and the issue is commonly referred to as “grade inflation”

Grade Inflation Economic Description – Grades Increasing Over Time (Dow Jones) Grade Inconsistent with Demonstrated Achievement – Student Assigned Grade of “A” with Limited Achievement – Does not Accurately Reflect Performance – Covering 9 versus 17 Chapters

The “Gold” Standard The goal is to create a common standard to compare students and grades assigned Use of standardized test information – ACT – CAAP “Rising Junior” Exams – GRE

Requested Information for Study Grades Assigned and Student Evaluations ( ) – Year – College – Department – Program – Section Example of Demographic Variables – First Year Enrolled Status (Transfer/Freshman) – High School GPA – Degree and Year Completed Note: 1992 – 2004 Represents the Time Frame Computerized Data Available

Federal Education Rights and Privacy ACT (FERPA) Institutional Review Board – Provided Approval Anonymous student and faculty identification numbers used Additional Protection: If sample size was less than 10 observations an any analysis level, information was not reported.

Methodology Checking the Data! Initial Calculations – Computing student GPAs independently – Correlated 1.0 with University reported GPAs Note: Computed with only fall and spring grades Modifications for Analysis Purposes – Conversion of grades to whole values for specific analyses Faculty Evaluations – Use of means in analysis of faculty evaluations

Research Design Exploratory Data Analytic Techniques – Tukey (1978) Graphing Descriptive Statistics Confidence Intervals Trend Analysis Layered Overview of Results – University -- College -- Department -- Program -- Course -- Section

Preliminary Results: Undergraduate U of A Grades from Mean GPAs increased from 2.76 to 2.95 during 12 year period

College Undergraduate GPA Trends with U of A GPA Trends for GPAs EDUC= 3.52 AFLS= 3.14 ENGR= 3.06 ARSC= 2.82 ARCH= 2.82 WCOB= 2.77

University and College GPA Trends 1992 – 2004: Rescaled Rescaling on 0.0 to 4.0 grade scale may create a different impression Rosenthal (1973), studies on interpreting graphs

Student Evaluations and Grades Assigned: Is there a pattern? The correlation between expected grade and university core average was.29, suggesting only a moderate association 86% of students expect grade of A or B Steady decline, then levels Based on the six Teaching Evaluation items all instructors must ask students!

What Represents an Example of Effective versus Ineffective Grading High or Low Grades? – Indicators of success? – Indicators of ineffective instruction? – Grade inflation? Patterns! – Identify patterns in the data that demonstrate cause for concern or discussion

Demonstrate a Process for Assessing the Assignment of Grades College of Education and Health Professions – Department of Educational Leadership, Counseling and Foundations Educational Foundations Program Graduate Degree Program Goal: To provide a method for evaluating the assignment of grades in a program area!

Graduate GPAs at the U of A 2004 GPAs EDUC= 3.77 ENGR= 3.70 ARSC= 3.66 AFLS= 3.65 WCOB= 3.55 Overall, graduate GPAs higher and have increased.

College of Education and Health Professions: Department Graduate GPAs Interesting! The GPAs by Dept. are becoming more variable

Educational Leadership, Counseling and Foundation Program Area Graduate GPAs Ed. Stat grades declining relative to rest of department However, not the complete picture!

Educational Foundations: Evaluating our Grading Patterns Tenure track faculty assign grades much lower than adjuncts in Ed. Stats Volatility due to one course

Implications of Internal Analysis We …. We the faculty in Educational Research and Policy Studies need to do a better job of overseeing/selecting our adjuncts. We …. We need to individualize the issue of grades, expectations, and assignment of grades. We …. We need to examine the content level and determine if expectations are commensurate with benchmark institutions. – Multiple Regression Example

Preliminary Conclusions Grades are increasing! – Is this problematic? Possible Explanations/Further Research – Higher entrance scores (ACT exams) – Transfer students – Transferring credits from other institutions (Community Colleges, Universities, etc.) – Academic expectations – Course taking patterns by students – Plus, many others ….

Preliminary Recommendations Stair-Step Evaluation of Grades Information Data Provided to Each College for Evaluations – Committees to examine information at each level and report -- College -- College Reports -- Department -- Department -- Program -- Program -- Course -- Course Evaluate-- Section  -- Section Ongoing Accountability

Next Steps Final modifications of requested data received on March 18 th. Preliminary analyses completed, more in- depth study of issues during the next four months to understand the implications. Comprehensive Report completed and submitted to Faculty Senate fall of 2005.