1. Measuring Teaching Practices
Reformed Teaching Observation Protocol1
Describes teaching process on five subscales
Lesson Design by instructor
Propositional knowledge of instructor
Procedural Knowledge (what students do)
Student-Student Interactions
Student-Teacher Interactions
Five items per subscale
Each item scored 0 (never occurred) to 4 (very descriptive of the class
Maximum Score = 100 (25 x 4)
Typical scores range from 20-80
Observer perspective
Requires trained observers
Quantitative (ordinal) data: 0-4 scale for each of the 25 items
0 (Never occurred) to 4 (Very descriptive of the class)
Qualitative data: Field notes
1Sawada et al., 2002

2. Measuring Teaching Practices
Reformed Teaching Observation Protocol1
18) There was a high proportion of student talk and a significant amount of it occurred between and among students (quantity of interactions)
No student-student talk
Students talk to each other at least once (about lesson content) 1
Student-student talk occurs at least 10% of the time during the course of the class 2
Student-student talk occurs more than 25% of the time during the course of the class 3
In any given moment during the lesson, students are more likely to be talking to each other than the teacher (>50% student to student) 4
Supplemental rubric created by Budd et al.
Greater inter-rater reliability per item.
Higher scoring classrooms feature more active learning practices.
Requires trained observers
Quantitative (ordinal) data: 0-4 scale for each of the 25 items
0 (Never occurred) to 4 (Very descriptive of the class)
Qualitative data: Field notes
1Sawada et al., 2002; Budd et al., 2013

3. Variations in Teaching Practices
Teaching Geoscience at US Higher Education Institutions
On the Cutting Edge has been using the Reformed Teaching Observation Protocol (RTOP) to measure teaching practices of more than 200 instructors in a variety of undergraduate geoscience classes at a range of US colleges and universities.
Here are some of the things we have discovered.

4. Variations in Teaching Practices
Teaching Geoscience at US Higher Education Institutions

5. RTOP Score vs. Sub-Categories
~all Faculty have high Propositional (Content) Knowledge
Best correlations with high RTOP score = STI, SSI, Procedural Knowl

6. 0.05 Small Medium Large Minimum 14 13 Maximum 78 89 69 Median 38.5 3834
First Quartile 26 31 22
Second Quartile
Third Quartile 53
47.5 45
Fourth Quartile
Small
Medium
Large -
0.83
0.05
0.10

7. Intro Majors 0.39 Intro Majors Label Total Score Minimum 13 16 Maximum89 78
Median 37
First Quartile
28.25 26
Second Quartile
Third Quartile 47 53
Fourth Quartile
Intro
Majors
0.39

8. Associate
Baccalaureate
Masters
Research/Doctoral
Minimum 20 17 13 14
Maximum 71 89 78 84
Median
43.5 36
39.5 35
First Quartile 32
31.25
32.25 25
Second Quartile
Third Quartile 52 44
49.5 47
Fourth Quartile
2yc
bacc
master
doctoral -
0.85
0.77
0.15
0.98
0.49
0.10

9. OtCE Participation vs. RTOP scores
How did participation in On the Cutting Edge events and use of web resources impact teaching practice?
Workshop only 42

10. Observed Teaching Practices
Most Traditional Lecture
n=10
Mean Traditional Lecture
Mean Transitional Lecture n=22
Mean Active Learning
n=12
Most Active Learning
n=11
No/few questions asked by instructor
50%
27% 0%
No/few questions from students
60%
36% 9%
Students are passive/not asked to do anything
70% 4%
No student-student interaction/ conversation
80%
32%
Student-student interactions or group work
59%
100%
91%
Students read graphs, maps, use data
20%
67%
45%
Students answer open-ended questions
17%
Instructor assesses students (new or prior knowledge)
10%
18%
33%
Lesson adjustments based on student work or prior knowledge
54%

11. Example Class Breakdowns
RTOP = 32, Transitional
Minutes 8 55 65
Activity 1
lecture
Activity 2
Activity 1: Present LOs, review with 3 conceptest questions
Activity 2: Group Venn diagram activity with shout out review
RTOP = 51, Student-Centered
Minutes 7 20 30 37 45 54 70 75
Activity 1
Lecture 1
Lecture 2
Activity 2
Lecture 3
Activity 3
Lecture 4
Act. 4
Activity 1: Present LOs and Brainstorming Think-Pair-Share
Activity 2: Group interpretive drawing activity with class discussion on implications
Activity 3: Activity connecting prior and common knowledge to content
Activity 4: Reflection, students summarize 3 most important concepts

12. InTeGrate Materials Development Process
Teams of authors spent 1-2 years creating new resources for their courses. They received instruction on pedagogy through workshops and webinars and feedback about their lessons from a team of coaches and assessment specialists.
Did it have an impact on their teaching practice?
InTeGrate materials are created by teams composed of three authors.
We have an approximately equal mix of participants for 2-year colleges, private institutions, and public 4-year schools that include instructors both masters and PhD granting departments.
One of the goals of the project is to create materials that can be used in all of these settings.
These materials have to meet the five overarching goals that are common to all InTeGrate projects.

13. Teaching Practice Data
(Reformed Teaching Observation Protocol; RTOP)
Student-centered
(83%)
56.2
41.5
Integrate pilot lessons (taught by the people who created the modules) typically scored in the student-centered range of RTOP.
Data represents multiple lessons from five modules taught by 11 instructors. Average RTOP scores from a range of intro courses were recorded by Budd et al at Average for InTeGrate lessons was much higher at 56.2.
Minerals
Climate EJ
Soils
Living on the Edge
Budd et al., 2013
Average RTOP score 41.5
InTeGrate pilot lessons
Average RTOP score 56.2

14. Conclusions:
RTOP data identifies 3 classroom types: Traditional Lecture, Active Lecture (Transitional), Active Learning
2. InTeGrate lessons taught by authors correlate with increased student engagement as measured by RTOP
3. Moving from less to more student engagement in class:
Increased use of questioning strategies for student input
Increased use of students working together during class
Increased discussion between faculty and students
Increased assessment (and adjustment) to student learning
An Active Learning class is very likely to have some traditional lecturing, along with having students participate in a brief group discussion, think-pair-share or class exercise during every class period.

15. 0.01 Adjunct Asst Assoc Full Inst/Lect Minimum 20 17 13 14 Maximum 6284 89 78 71
Median 31 42
42.5 37
34.5
First Quartile
23.5
32.5 26
24.5
Second Quartile
Third Quartile 44
55.5
53.75 50
41.75
Fourth Quartile
Adjunct
Asst
Assoc
Full
Inst -
0.05
0.04
0.21
0.98
0.87
0.02
0.18
0.01
0.06