1. CHRONOGICAL AND PEDAGOGICAL ANALYSIS OF GLOBAL REAL-TIME ASSESSMENT TOOL FOR TEACHING ENHANCEMENT (G-RATE) Syafiah M. Johari, Nikitha Sambamurthy, and Dr. Monica F. Cox International Institute for Engineering Education Assessment | Purdue University, West Lafayette, IN
Introduction
Results
Comparative Analysis : Chronological and Pedagogical Analysis from Observation Data Collected in Spring 2014
Global Real-Time Assessment Tool for Teaching Enhancement (G-RATE) serves as a pedagogical feedback tool to assess teaching performance in learning environments1
ORG
Organization C1
Engaged students in Team Activities C2
Provided opportunities for Students to Learn from Each Other C3
Helpeds teams when They Needed Assistance L1
Acknowledged that Learning Engineering Concepts Can Be Hard At Times L2
Acknowledged Students' Misunderstanding of Concepts L3
Provided Guidance for Students During Problem-Solving A1
Asked Thought-Provoking Questions A2
Provided Verbal or Written Feedback to Students about Their Progress or Performance A3
Confirmed the Class Understood Contents before Moving to A New Topic K1
Shared His/Her Own Practical Experiences K2
Related Course Contents to Everyday Situations K3
Shared Skills Students Can Apply Later K4
Emphasized Learning New Skills K5
Helped Students to Understand Key Concepts
G-RATE is developed based on the How People Learn (HPL) framework (29 items) which specifically identifies four dimensions which are essential elements of an effective learning environment 2 :
Community-centeredness - Connect students with the community in class.
Learner-centeredness - Attempt to connect students with their prior knowledge and/or experiences.
Assessment-centeredness - Provide activities for learners to make their thinking visible and provide feedback.
Treatment Group in Round 1
Control Group 2 in Round 1
Control Group 1 in Round 1
Treatment Group in Round 2
Control Group 2 in Round 2
Control Group 1 in Round 2
Knowledge-centeredness - Help students to develop understanding without a domain of knowledge.
Problem Statement
There are no succinct and meaningful representations for frequency and chronological observational data.
Figure 2 : Chronological Analysis Comparison Chart
No interpretation from the collected observation data has been made so far.
Observation data is compared from the first 10 seconds of observation until the last 40 minutes for both Round 1 and Round 2 in Spring 2014.
More activities carried out in the beginning of the observation in organization and community-centeredness.
Methods
Observation data collected from previous semester (Spring 2014) is compared between treatment and control groups to see the change in patterns (N=994)
Chronological analysis – observation data is compared based on the timestamp, each time slot differs by 10 seconds
Pedagogical analysis – observation data is compared based on the overall teaching performance throughout the whole observation period among all instructors
Figure 3 : Pedagogical Analysis in Terms of Percentage of Occurrence Comparison Charts
Observation data collected is compared among instructors by observing the frequency pattern for each sections in terms of percentage of occurrence throughout the observation period.
No comparison can be made for L1, K1, K3, and K4 – instructors did not emphasize these activities in class.
Figure 1 : Observer function of G-RATE Online Tool
Conclusions
Acknowledgements
Overall, for both chronological and pedagogical analysis, it could be observed that instructors perform better in organization and community-centered activities in Round 2 as compared to Round 1 from the beginning till the end of observation.
More activities in organization and community-centeredness might be because the observation is held in engineering class, which emphasizes teaming skills most of the time.
However, the pattern for other dimensions (learner-centeredness, knowledge-centeredness and assessment-centeredness) could not be predicted since it varies for each dimension.
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No
Any opinion, findings, and conclusions or recommendations expressed in this material are those of the authors(s) and do not necessarily reflect the views of the National Science Foundation.
Contact Information
References
Syafiah M. Johari
School of Mechanical Engineering, Purdue University.
Research is done under Dept. of Engineering Education.
Visit i2e2a.org for more information.
1Cox, M.F., Hahn, J., McNeill, N., Cekic, O., Zhu, J, & London, J. (2011), Enhancing the Quality of Engineering Graduate Teaching Assistants through Multidimensional Feedback, Advances in Engineering Education. 2 (3), 1-20.
2Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press, 1999.