ISTE Workshop Research Methods in Educational Technology IIT Bombay February 2-9, 2013

Session 5: Delving into RM-ET

5a: Introduction to research design

Recall – This Workshop is a Journey …   Recall – This Workshop is a Journey … From ET Practitioner teach students; facilitate their learning. come up with ideas for doing the above. To ET Researcher conduct systematic studies to get data about whether our ideas are working. provide evidence to support our conclusions.

Recall – This Workshop is a Journey …   Recall – This Workshop is a Journey … From ET Practitioner teach students; facilitate their learning. come up with ideas for doing the above. To ET Researcher conduct systematic studies to get data about whether our ideas are working. provide evidence to support our conclusions.

Our Goals as ET Researchers   Our Goals as ET Researchers conduct systematic studies to get data about whether our ideas are working. provide evidence to support our conclusions. This will establish soundness of your study.

What a novice ET researcher might do   What a novice ET researcher might do Start: Problem I am trying to solve Propose: My strategy to solve the problem Goal: Conduct systematic study to get data on whether my strategy is working.

What a novice ET researcher might do   What a novice ET researcher might do Start: Problem I am trying to solve Propose: My strategy to solve the problem Goal: Conduct systematic study to get data on whether my strategy is working. Method: Step 1 – Implement strategy in class Step 2 – Conduct a test to check how well students have done after learning with my strategy Step 3 – If students get high marks on test, claim that my strategy works

Will this method establish that my   Vote Method: Step 1 – Implement strategy in class Step 2 – Conduct a test to check how well students have done after learning with my strategy Step 3 – If students do well on test, claim that my strategy works Will this method establish that my strategy works? Yes No Participants – please vote! Co-ordinators – how many Yes, how many No – convey by chat.

What will the referee say?   What will the referee say? Method: Step 1 – Implement strategy in class Step 2 – Conduct a test to check how well students have done after learning with my strategy Step 3 – If students do well on test, claim that my strategy works Will this method establish that my strategy works? Yes No Referee decision – Paper Rejected. Method not sound enough to establish claim.

Think-Pair- Share activity   Think-Pair- Share activity A novice researcher may do the following: Implement strategy in class; conduct a test to check how students have done after learning with strategy. If students get high marks, claim that the strategy works. Question: What are possible reasons this method is not the best to establish that ‘my strategy works’? THINK: Write one reason why the above method is not sound. PAIR: Share your reason with your partner.

Check answers: yours & your partner’s   Check answers: yours & your partner’s SHARE: If you or your partner thought of a reason below, raise your hand. Implementing my strategy for a single group, and conducting a post-test only does not necessarily mean that my strategy works, because: The result could have occurred due to a reason other than my strategy. There is no comparison to a group that did not learn using my strategy The result might have existed even before the strategy was implemented. (for ex: students may already have knowledge of the topic)

Why is single-group post-test only research design not sound?   Why is single-group post-test only research design not sound? The outcome could have occurred due to a reason other than the treatment There is no comparison to a group that did not receive the treatment The outcome could have existed even before the treatment occurred There is no control of other possible influences on the outcome

How to design a sound, systematic ET research study   How to design a sound, systematic ET research study Let us see an example first. Developing engineering design competency. Madhuri Mavinkurve Faculty member, Thakur College of Engg., Mumbai Ph.D. student, Educational Technology, IIT Bombay