1. Dr John Willison Centre for Learning and Professional Development University of Adelaide Developing and Assessing Students’ Research Skills in the Curriculum A seminar for the Faculty of Economics and Commerce, University of Melbourne 2 Australian Learning and Teaching Council

2. Overview  Analyse collaboratively, and draw own conclusions about, the Research Skill Development (RSD) framework.  Examine examples from Business and disciplines that have used wikis  Apply the RSD to a context of relevance

3. Disciplines Involved in Trialling RSD Business (Melbourne and Monash) Dentistry/ Oral Health, Electronic Engineering, Introduction to Academic Learning for International Students, Medical Science, Nursing, Petroleum Engineering (Uni of Adelaide) Psychology (Macquarie) Introduction to Tertiary Learning (University of South Australia) Impending: Chemical Engineering and Computing Science In train: VU (Engineering), QUT, USQ (Multiple)

4. Why Research Skill Development (RSD) in the curriculum? Undergraduate research experiences are claimed to increase student motivation, satisfaction, understanding of content, progression to, and completion of HDR, and to reduce attrition and plagiarism. But… Without the modeling of research practices at a higher level of thinking students continue to engage in investigations at the introductory level, even though their exposure to content is considerably richer with each course they take. (Chaplin, 2003 p.238)

5. Research Skill Development in Curriculum Design and Assessment In 12+ Disciplines so far, changes that lectures have made are small Lecturers take existing assessment, and rework the marking criteria according to the RSD Use analogous criteria eg Oral Health used Business Some missing facets e.g. ‘evaluate’, are incorporated ‘Diagnostic’ Assessments have been changed or added Otherwise, the nature of assessments is unchanged The focus of the course is either renewed or clarified Explicit nature of RSD changes curriculum

6. Level I closed inquiry structured and guided Level II closed inquiry, structured Level III closed inquiry independently Level IV open inquiry, structured guidelines Level V open inquiry Levels of Student Autonomy FacetsFacets Students: A. Embark, determining need for knowledge B. Find/generate C. Critically evaluate D. Organise information E. Synthesise, analyse, apply F. Communicate Research Skill Development Framework

8. Assessments Based on the RSD framework In Business Courses (see pages 58-61 of RSD Handbook) I II III IV Open-ended Research I I II III A···FA···F A···FA···F A···FA···F Entry Level RSD Lit RSD 3 I II III A···FA···F Entry Level RSD Business Law Business Ethics

9. Life Impact The University of Adelaide Application of RSD Framework for Learning and Teaching Slide 9

10. RSD Framework Provides the Big Picture and relates this to the assessment details for course coordinators, lecturers, tutors, and especially students Informs assessment-first curriculum redesign Same ‘facets’ for multiple assessments, various levels Explicit &Transparent assessment criteria Coherent & Incremental skill development Revisited & (potentially) Cyclic Conceptual Framework

11. References Bloom, B., Engelhardt, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R (1956). Taxonomy of Educational Objectives, NewYork: David McKay Company. Australian and New Zealand Institute of Information Literacy (2004). Australian and New Zealand Information Literacy Framework: Principles, standards and practice (2nd Edition). Retrieved from http://www.caul.edu.au/info-literacy/InfoLiteracyFramework.pdf on 15/4/08. http://www.caul.edu.au/info-literacy/InfoLiteracyFramework.pdf on 15/4/08 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edition). Hillsdale, NJ: Erlbaum. Chaplin, S.B. (2003). Guided development of independent inquiry in an anatomy/physiology laboratory. Advanced Physiology Education 27: 230-240, 2003 Willison, J.W. & O’Regan, K. (2007). Commonly known, commonly not known, totally unknown: A framework for students becoming researchers. Higher Education Research and Development 26 (4). Web Site www.adelaide.edu.au/clpd/rsd Lots of downloadable RSD examples are available here.

12. Acknowledgement Support for this activity has been provided by The Australian Learning and Teaching Council, an initiative of the Australian Government Department of Education, Science and Training. The views expressed in this activity do not necessarily reflect the views of Australian Learning and Teaching Council. ‘Application of RSD’ slides were provided by Dr Mario Ricci and Dr Eleanor Peirce, University of Adelaide For more Information… … or to discuss ideas further, contact John john.willison@adelaide.edu.au 08 8303 3219

13. r= 0.57

14. Correlations between RSD Assessments 2005-2007 200520062007 O week vs Lit RSD 1 0.18 n=98 0.13 n=119 0.01 n=138 O week vs Open Inquiry 0.16 n=82 0.09 n=97 0.04 n=95 Lit RSD 1 vs Lit RSD 3 N/A 0.14 N=115 0.30 n=121 Pwer = 0.95, p=0.05

15. Correlations between RSD Assessments 2005-2007 200520062007 Lit RSD 1 vs Open Inquiry 0.41 n=82 Power = 0.99, p=0.05 0.30 n=97 Power = 0.91, p=0.05 0.38 n=95 Power = 0.96, p=0.05 Lit RSD 3 vs Open Inquiry 0.48 n=79 (Power=0.98 p<0.01) 0.55 n=97 (power>0.995, p<0.01) 0.57 n=95 (power>0.995, p<0.01)

16. Correlations between RSD and pre-RSD Assessments 2003-2007 2003 (pre- RSD) 2004 (pre- RSD) 2005 (mid RSD) 20062007 Lit RSD 1 vs Open Inquiry N/A 0.41 n=82 Power = 0.99, p=0.05 0.30 n=97 Power = 0.91, p=0.05 0.38 n=95 Power = 0.96, p=0.05 Lit RSD 3 vs Open Inquiry 0.20 n= 104 Power 0.77 p < 0.1 0.40 n=117 Power 0.99 p< 0.01 0.48 n=79 Power>0.98 p<0.01 0.55 n=97 power>0.99 p<0.01 0.57 n=95 power>0.99 p<0.01

17. Facets associated with research processes In researching, students: A.embark on an inquiry and so determine a need for knowledge/understanding B.find/generate needed information/data using appropriate methodology C.critically evaluate information/data and the process to find/generate D.organise information collected/generated E.synthesise and analyse and apply new knowledge F.communicate knowledge and the processes used to generate it, with an awareness of ethical, social and cultural issues. (Willison & O’Regan, 2007; based on Bundy, 2004).

18. Levels of Student Autonomy Level I Students research at the level of a closed inquiry and require a high degree of structure/guidance Facets of research inquiry A. Students embark on inquiry and so determine a need for knowledge/ understanding B. Students find/generate needed information C. Students critically evaluate information /data and the process to find/generate D. Student organise information collected/generated E. Students synthesise and analyse and apply new knowledge F. Students communicate knowledge and understanding and the process used to generate them. Level I Student researches at the level of a closed inquiry and requires a high degree of structure/guidance Appropriately identifies key ideas from 1 source. When using 2 or more sources, identifies some peripheral or duplicated ideas as key Significance of the paper is stated, but not based on leads from, or gaps in, the literature http://www.adelaide.edu.au/clpd/rsd/cells/a/a1.html

19. Levels of Student Autonomy Level II Students research at the level of a closed inquiry and requires structure Facets of research inquiry A. Students embark on inquiry and so determine a need for knowledge/ understanding B. Students find/generate needed information C. Students critically evaluate information /data and the process to find/generate D. Student organise information collected/generated E. Students synthesise and analyse and apply new knowledge F. Students communicate knowledge and understanding and the process used to generate them. Level I Student researches at the level of a closed inquiry and requires a high degree of structure/guidance Level II Student researches at the level of a closed inquiry and requires some structure/guidance Identifies key ideas across several sources Significance of the paper is stated explicitly and is based on leads from, or gaps in, a limited number of references http://www.adelaide.edu.au/clpd/rsd/cells/a/a2.html

20. Levels of Student Autonomy Level III Students research independently at the level of a closed inquiry Facets of research inquiry A. Students embark on inquiry and so determine a need for knowledge/ understanding B. Students find/generate needed information C. Students critically evaluate information /data and the process to find/generate D. Student organise information collected/generated E. Students synthesise and analyse and apply new knowledge F. Students communicate knowledge and understanding and the process used to generate them Level I Student researches at the level of a closed inquiry and requires a high degree of structure/guidance Level II Student researches at the level of a closed inquiry and requires some structure/guidance Level III Student researches independently at the level of a closed inquiry Significance of the paper is clearly stated and based on leads from, or gaps in, a substantial number of sources http://www.adelaide.edu.au/clpd/rsd/cells/a/a3.html

21. Levels of Student Autonomy Level IV Students research at the level of an open enquiry within structured guidelines Level I Student researches at the level of a closed inquiry and requires a high degree of structure/guidance Level II Student researches at the level of a closed inquiry and requires some structure/guidance Level III Student researches independently at the level of a closed inquiry Level IV Student researches at the level of an open enquiry within structured guidelines Facets of research inquiry A. Students embark on inquiry and so determine a need for knowledge/ understanding B. Students find/generate needed information C. Students critically evaluate information /data and the process to find/generate D. Student organise information collected/generated E. Students synthesise and analyse and apply new knowledge F. Students communicate knowledge and understanding and the process used to generate them Aims/hypothesis clear, focussed and innovative http://www.adelaide.edu.au/clpd/rsd/cells/a/a4.ht ml

22. Levels of Student Autonomy Level V Students research at the level of an open inquiry within self-determined guidelines Facets of research inquiry A. Students embark on inquiry and so determine a need for knowledge/ understanding B. Students find/generate needed information C. Students critically evaluate information /data and the process to find/generate D. Student organise information collected/generated E. Students synthesise and analyse and apply new knowledge F. Students communicate knowledge and understanding and the process used to generate them Level I Student researches at the level of a closed inquiry and requires a high degree of structure/guidance Level II Student researches at the level of a closed inquiry and requires some structure/guidance Level III Student researches independently at the level of a closed inquiry Level IV Student researches at the level of an open enquiry within structured guidelines Level V Student researches at the level of an open enquiry within self- determined guidelines http://www.adelaide.edu.au/clpd/rsd/cells/a/a5.html

23. Dr John Willison Centre for Learning and Professional Development University of Adelaide Locating the Development and Assessment of Students’ Research Skills in the Curriculum A workshop for the Faculty of Business and Economics, Monash University 2 Australian Learning and Teaching Council

24. Workshop Objectives  Develop collaboratively a plan of action for Research Skill Development in a specific course Or  Develop specific marking criteria for a particular assessment task Or  Another RSD-based possibility that has come to mind