CHRONOGICAL AND PEDAGOGICAL ANALYSIS OF GLOBAL REAL-TIME ASSESSMENT TOOL FOR TEACHING ENHANCEMENT (G-RATE) Syafiah M. Johari, Nikitha Sambamurthy, and Dr. Monica F. Cox International Institute for Engineering Education Assessment | Purdue University, West Lafayette, IN Introduction Results Comparative Analysis : Chronological and Pedagogical Analysis from Observation Data Collected in Spring 2014 Global Real-Time Assessment Tool for Teaching Enhancement (G-RATE) serves as a pedagogical feedback tool to assess teaching performance in learning environments1 ORG Organization C1 Engaged students in Team Activities C2 Provided opportunities for Students to Learn from Each Other C3 Helpeds teams when They Needed Assistance L1 Acknowledged that Learning Engineering Concepts Can Be Hard At Times L2 Acknowledged Students' Misunderstanding of Concepts L3 Provided Guidance for Students During Problem-Solving A1 Asked Thought-Provoking Questions A2 Provided Verbal or Written Feedback to Students about Their Progress or Performance A3 Confirmed the Class Understood Contents before Moving to A New Topic K1 Shared His/Her Own Practical Experiences K2 Related Course Contents to Everyday Situations K3 Shared Skills Students Can Apply Later K4 Emphasized Learning New Skills K5 Helped Students to Understand Key Concepts G-RATE is developed based on the How People Learn (HPL) framework (29 items) which specifically identifies four dimensions which are essential elements of an effective learning environment 2 : Community-centeredness - Connect students with the community in class. Learner-centeredness - Attempt to connect students with their prior knowledge and/or experiences. Assessment-centeredness - Provide activities for learners to make their thinking visible and provide feedback.   Treatment Group in Round 1 Control Group 2 in Round 1 Control Group 1 in Round 1 Treatment Group in Round 2 Control Group 2 in Round 2 Control Group 1 in Round 2 Knowledge-centeredness - Help students to develop understanding without a domain of knowledge. Problem Statement There are no succinct and meaningful representations for frequency and chronological observational data. Figure 2 : Chronological Analysis Comparison Chart No interpretation from the collected observation data has been made so far. Observation data is compared from the first 10 seconds of observation until the last 40 minutes for both Round 1 and Round 2 in Spring 2014. More activities carried out in the beginning of the observation in organization and community-centeredness. Methods Observation data collected from previous semester (Spring 2014) is compared between treatment and control groups to see the change in patterns (N=994) Chronological analysis – observation data is compared based on the timestamp, each time slot differs by 10 seconds Pedagogical analysis – observation data is compared based on the overall teaching performance throughout the whole observation period among all instructors Figure 3 : Pedagogical Analysis in Terms of Percentage of Occurrence Comparison Charts Observation data collected is compared among instructors by observing the frequency pattern for each sections in terms of percentage of occurrence throughout the observation period. No comparison can be made for L1, K1, K3, and K4 – instructors did not emphasize these activities in class. Figure 1 : Observer function of G-RATE Online Tool Conclusions Acknowledgements Overall, for both chronological and pedagogical analysis, it could be observed that instructors perform better in organization and community-centered activities in Round 2 as compared to Round 1 from the beginning till the end of observation. More activities in organization and community-centeredness might be because the observation is held in engineering class, which emphasizes teaming skills most of the time. However, the pattern for other dimensions (learner-centeredness, knowledge-centeredness and assessment-centeredness) could not be predicted since it varies for each dimension. This material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. 1140763. Any opinion, findings, and conclusions or recommendations expressed in this material are those of the authors(s) and do not necessarily reflect the views of the National Science Foundation. Contact Information References Syafiah M. Johari <sjohari@purdue.edu> School of Mechanical Engineering, Purdue University. Research is done under Dept. of Engineering Education. Visit i2e2a.org for more information. 1Cox, M.F., Hahn, J., McNeill, N., Cekic, O., Zhu, J, & London, J. (2011), Enhancing the Quality of Engineering Graduate Teaching Assistants through Multidimensional Feedback, Advances in Engineering Education. 2 (3), 1-20. 2Bransford, J. D., Brown, A. L., & Cocking, R. R. (Eds). How People Learn: Brain, Mind, Experience, and School. Washington, DC: National Academy Press, 1999.