1. ISTE Workshop Research Methods in Educational Technology
IIT Bombay
February 2-9, 2013

2. Session 5:
Delving into RM-ET

3. 5a: Introduction to research design

4. Recall – This Workshop is a Journey …
Recall – This Workshop is a Journey …
From
ET Practitioner
teach students; facilitate their learning.
come up with ideas for doing the above. To
ET Researcher
conduct systematic studies to get data about whether our ideas are working.
provide evidence to support our conclusions.

5. Recall – This Workshop is a Journey …
Recall – This Workshop is a Journey …
From
ET Practitioner
teach students; facilitate their learning.
come up with ideas for doing the above. To
ET Researcher
conduct systematic studies to get data about whether our ideas are working.
provide evidence to support our conclusions.

6. Our Goals as ET Researchers
Our Goals as ET Researchers
conduct systematic studies to get data about whether our ideas are working.
provide evidence to support our conclusions.
This will establish soundness of your study.

7. What a novice ET researcher might do
What a novice ET researcher might do
Start: Problem I am trying to solve
Propose: My strategy to solve the problem
Goal: Conduct systematic study to get data on whether my strategy is working.

8. What a novice ET researcher might do
What a novice ET researcher might do
Start: Problem I am trying to solve
Propose: My strategy to solve the problem
Goal: Conduct systematic study to get data on whether my strategy is working.
Method:
Step 1 – Implement strategy in class
Step 2 – Conduct a test to check how well students have done after learning with my strategy
Step 3 – If students get high marks on test, claim that my strategy works

9. Will this method establish that my
Vote
Method:
Step 1 – Implement strategy in class
Step 2 – Conduct a test to check how well students have done after learning with my strategy
Step 3 – If students do well on test, claim that my strategy works
Will this method establish that my
strategy works?
Yes No
Participants – please vote!
Co-ordinators – how many Yes, how many No – convey by chat.

10. What will the referee say?
What will the referee say?
Method:
Step 1 – Implement strategy in class
Step 2 – Conduct a test to check how well students have done after learning with my strategy
Step 3 – If students do well on test, claim that my strategy works
Will this method establish that my
strategy works?
Yes No
Referee decision –
Paper Rejected.
Method not sound enough to establish claim.

11. Think-Pair- Share activity
Think-Pair- Share activity
A novice researcher may do the following:
Implement strategy in class; conduct a test to check how students have done after learning with strategy. If students get high marks, claim that the strategy works.
Question: What are possible reasons this method is not the best to establish that ‘my strategy works’?
THINK: Write one reason why the above method is not sound.
PAIR: Share your reason with your partner.

12. Check answers: yours & your partner’s
Check answers: yours & your partner’s
SHARE: If you or your partner thought of a reason
below, raise your hand.
Implementing my strategy for a single group, and conducting a post-test only does not necessarily mean that my strategy works, because:
The result could have occurred due to a reason other than my strategy.
There is no comparison to a group that did not learn using my strategy
The result might have existed even before the strategy was implemented. (for ex: students may already have knowledge of the topic)

13. Why is single-group post-test only research design not sound?
Why is single-group post-test only research design not sound?
The outcome could have occurred due to a reason other than the treatment
There is no comparison to a group that did not receive the treatment
The outcome could have existed even before the treatment occurred
There is no control of other possible influences on the outcome

14. How to design a sound, systematic ET research study
How to design a sound, systematic ET research study
Let us see an example first.
Developing engineering design competency.
Madhuri Mavinkurve
Faculty member, Thakur College of Engg., Mumbai
Ph.D. student, Educational Technology, IIT Bombay