1. 1 Correlating Knowledge of Landscape Formation Timescales and Geologic Time using a New Validated Concept Test. ~ Dept Earth and Ocean Sciences Alison Jolley, Francis Jones, Sara Harris, Jamil Rhajiak Earth and Ocean Sciences, UBC.

2. 2 Outline 1.Context 2.Geotime concepts 3.Landscape Identification and Formation Test (LIFT) 4.Results & implications 5.Lessons learned, and questions

3. 3 Context Concept tests: used increasingly in several disciplines – To measure learning gains (in a course or module) – To compare students, pedagogies, etc. 2008: GeoTime test developed – Concept test goal: measure learning gains in EOS. – Student attitude survey about Earth & Ocean Sciences 2009: L.I.F.T. developed, focusing on geomorphic rates and processes.

4. 4 Geotime concept test development Experts: Interviewed to identify topics of interest. – 20 questions produced. Students: Validation via iterative think-aloud interviews. The Test: Range of concepts & Blooms level coverage. Key ConceptsGeoT Timescale3 Relative dating6 Absolute dating4 Earth history6 Uniformitarianism1 Processes & rates0 Bloom's levelGeoT knowledge8 comprehension3 application5 analysis4 (Honors thesis, Rhajiak, 2009) # Qns After using GeoTime in a 3 rd /4 th yr elective course, eight questions were selected for use in L.I.F.T. 341321341321

5. 5 Landscapes concept test development Select landscapes Develop questions Student think-aloud interviews Expert interviews Generate answer key Deliver LIFT Iterative revision (Honors thesis, Jolley, 2010) Questions first. Student interviews crucial for validation. Expert interviews used to generate ranges of acceptable answers.

6. 6 Expert results for answer ranges: 12 landscapes: seconds < T < 10 8 yrs. 7 experts; Correct selected from the range of consensus. Incorrect added outside that range. Spread of expert answers increased for middle time ranges.

7. 7 LIFT question sequence: 1.Image projected for 45 seconds. Students answer the following: 2.Next image is shown (total of 12 images used). 3.Eight Geo-Time questions at the end. Image Copyright © Marli Miller, University of Oregon; Image source: Earth Science World Image Bank; http://www.earthscienceworld.org/images ID Confidence Timescale

8. 8 Initial LIFT deployment Two geoscience courses: 2 nd yr. and 4 th yr. (N = 71 and 25). Geoscience majors (9 not geoscientists in 4 th yr). 30 mins. / paper-based / marked by the researcher. Demographics recorded: gender, age, major, prior geology courses taken.

9. 9 LIFT results: by class vs. by experience Diverse geoscience classes: class year NOT a good indicator of geoscience ability. Correlation between knowledge of geologic time and landscape formation times ALL students: - Good at recognizing landscapes. - LESS good at estimate formation times. Lower scores seem overconfident. (more later…)

10. 10 LIFT results: right / wrong sorted by timescale Time scales not well recognized. Locally familiar landscapes recognized better. Very short and very long timescales known better. Experts also were variable with intermediate timescales. What implications for geo-science classes ??? IDvs. Formation timescales

11. 11 ConfidentNOT advanced 70% 9% beginners 58% 23% Confidence in formation times (correct landscape ID). Beginners: confidence is flatter, regardless of right or wrong. Do beginners metacognitive skills improve simply with more practice? Advanced: more certain than beginners when right.

12. 12 LIFT results – gender (all students) (preliminary) I.D. All roughly equal at I.D. and confidence in ID. timescale All have roughly equal timescale scores. MALES seem more confident about timescales. FEMALES seem to have a larger std. errors.

13. 13 Geological time scores by gender & prior knowledge: Evidently, at 4 th yr level: Little effect of gender, Some effect of prerequisite. Evidently, at 2 nd yr level: Slight effect of gender. Nil effect of major. Regarding knowledge of geological time:

14. 14 Comments so far: These data provide an initial window into … – Comparing types of knowledge about geological time – Degree of agreement among experts about landscapes – Impact of students background in diverse courses – Development of metacognition (confidence) in the context of geological time and processes – Priorities for teaching various types of knowledge & skills Lessons: assessing geoscience learning … – Is context dependent – Requires various types of validated instruments – Is challenging

15. 15 Conclusions Both theses at : https://circle.ubc.ca/handle/2429/23321 https://circle.ubc.ca/handle/2429/23321 Questions and discussion Thanks to: -Students who participated. -Faculty who contributed time, advice and expertise. -Colleagues with the Carl Wieman Science education Initiative. -Colleagues at Colorado University.