1. Heidi Diefes-Dux Adam Carberry Matthew Siniawski Sara Atwood
FOEE September 27, 2016
Data-Driven Instruction using Standards-Based Grading (SBG)
Heidi Diefes-Dux
Adam Carberry
Matthew Siniawski
Sara Atwood

2. My Experience with Data-Driven Instruction
First-Year Engineering (N=1500+)
Model-Eliciting Activities & Design Challenges
Standards-Based Grading (SBG) for 4 years
Student and grader data used to improve:
Problem design
Assessment strategies (e.g. learning objective clarification, rubric design, and feedback)
Student instruction
Instructor and TA training

3. Formative Assessment
Systematic way for teachers and students to gather evidence of learning and use data to improve teaching and learning
Purpose is to close the gap between expected and actual learning

4. assess students’ achievement of these?
ABET
Program Outcomes – broad statements of significant and essential learning that should be demonstrated upon graduation
Course Goals – broad statements of what students should be able to do by the end of the course
Course Learning Objectives – specific criterion-referenced observable behaviors (tied to individual units of instruction)
But do we explicitly
assess students’ achievement of these?

5. Standards-based Grading (SBG)
Assessment of student’s mastery of essential standards or learning objectives for a course
Provides clear, meaningful, and personalized feedback
Directly connects students’ development with specific objectives (or standards)
Promotes learning and encourages continuous improvement
Provides fairness and transparency in the grading process
Provides evidence for course and program assessment
Sadler, 2005; Tomlinson & McTighe, 2006; Carpinelli et al., 2008; Scriffiny, 2008

6. Standards-Based Analytic Rubric
LO02.00 Create and evaluate a x-y plot suitable for technical presentation
No Evidence
Underachieved
Partially Achieved
Fully Achieved
More than 4 errors from Fully Achieved list
3-4 errors from Fully Achieved List
1-2 errors from Fully Achieved list
Plot command used
Independent & dependent variable on correct axis
Plot formatted correctly (markers, lines, legend)
Hold or equivalent is used
Appropriate title
Appropriate axis labels (with units)

7. Problem
Use MATLAB to create a single plot with the populations of Rabbits versus Time and Wolves versus Time.

8. Rubric Item Applied to Problem
Fully Achieved
Plot command used
Independent & dependent variable on correct axis
Plot formatted correctly (markers, lines, legend)
Hold or equivalent is used
Appropriate title
Appropriate axis labels (with units) ✓ ✓ ✓ ✓ ✓ X

9. Rubric Item Applied to Problem
Partially Achieved
1-2 errors from Fully Achieved ✓

10. Overall Assessment of Learning Objective
LO02.00 Create and evaluate a x-y plot suitable for technical presentation
Assessment %
No Evidence
Under-achieved
Partially Achieved
Fully Achieved
Problem Set 01 2% 3%
23%
71%
Problem Set 02 9% 5%
27%
58%
Exam 1
18%
51%
26%

11. Identification of Variability Across Sections
LO GPA
Learning Objective
15.5
15.7
Assignment
PS07
PS08
Section A
3.38
3.77
2.68
Section B
3.47
3.73
3.80
Section C
3.79
3.92
3.03 …
Section G
2.21
3.14
0.00
Section H
3.57
3.71
1.48
Section I
3.84
3.65
3.51
All Sections:
3.32
3.52
2.71
Leads to Improved Problems, Rubrics, and Training

12. Me: F TA: B Me: C/D TA: A Me: A TA: C/D Me: C/D TA: C/D
LO Appropriately use the best-fit linear model to make predictions
F = No Evidence
C/D = Under
B = Partially
A = Fully Achieved
Anything less than requirements for underachieved
Any 1 elements missing or incorrect from the list for fully achieved
Not Used for Grading
Problem 2
Predicts aspirin yield for wintergreen values within range of original data set
Acknowledges wintergreen values outside range of data cannot be used for predicting aspirin yield
Answer key solution
𝒚=𝟎.𝟐𝟗𝒙+𝟎.𝟏𝟑
Student 3 Solution:
y= x (Note: Their graph was y = .2939x )
For 9 g:
2.5333g (Their equation yields 3.797)
For 12 g:
g (Their equation yields 4.747)
For 16 g:
4.75 g (Their equation yields 6.014)
For 30 g:
g (Their equation yields )
Me: F
TA: B
For 9 g:
2.78g, assuming the linear function holds outside the data range
For 12 g:
3.66 g
For 16 g:
4.84 g
For 30 g:
8.95 g, assuming the linear function holds outside the data range
Student 4 Solution:
y=0.2939x
For 9 g:
This is not an accurate prediction because 9g is outside of the given data set used for the prediction.)
For 12 g:
3.6594
For 16 g:
4.8354
For 30 g:
(This is not an accurate prediction because 30g is outside of the given data set used for the prediction.)
Me: A
TA: C/D
Student 1 Solution:
0.2939x
For 9 g:
2.779 g of aspirin
For 12 g:
g of aspirin
For 16 g:
g of aspirin
For 30 g:
g of aspirin
Student 2 Solution:
y=0.2939x
For 9 g:
Not in model range, could not b computed
For 12 g:
0.472 g
For 16 g:
4.836 g
For 30 g:
8.951 g
Me: C/D
TA: A
Me: C/D
TA: C/D

13. What can we learn from using SBG?
We can identify the most difficult learning objectives and react in real time or improve for next time

14. We can share best practices
Identify leading sections
Identify lagging sections

15. Next Steps Best practices and barriers for SBG
Grader training strategies
Enable easier post-processing of LO data
Better post-assignment class discussions
Quicker identification of grader and instructional challenges and successes
Student dashboard for easier LO performance tracking

16.
Making Grades Meaningful - Standards-based Grading for Engineering Project Courses, Grant No. DUE