1. Tablets for Medical Education (T4ME) in Rwanda: Usage Patterns & Outcomes Over Three Years Distance Teaching & Learning Conference, Madison, WI Aug 13, 2015 Adam Papendieck MPH Program Director, Tulane University School of Law, Payson Graduate Program in Global Development Doctoral Student, University of Texas at Austin, C&I - Learning Technologies Nancy Mock DrPH Assistant Professor, Tulane University, School of Public Health and Tropical Medicine Patrick Kyamanywa MD Associate Professor, University of Rwanda, School of Medicine

3. Tablets for Medical Education (T4ME) In fall 2012, 91 third year medical students were given 7-inch Samsung Galaxy tabs with a case, keyboard, storage, sim card (2 months data) Preloaded with a variety of apps, e.g.: Epocrates Medscape DropBox Kindle Annotation Skype Support Training Facebook User Groups Email support

4. Goals 1.Increase access to internet and (digital) resources 2.Network a community of practice: improve connectivity, communication and collaboration 3.Enhance new literacies and digital participation (Jenkins et al., 2006) 4.Improve learning outcomes Photo source: The New Times / JP Bucyensenge

5. Outcomes: Internet and Device Usage N=91 n=86 n=82 n=21

6. Outcomes: Centrality of Web N=91 n=86 n=82 n=21

7. Outcomes: Resource Access N=91 n=86 n=82 n=21

8. 2012 Test Cohort vs. 2011 (Control) *all differences were significant at p<.000 (86 out of 91 responding)

9. The Angry Birds Hypothesis

11. Type of usage and exam scores No significant association between overall scores and usage patterns A few significant negative associations with individual exam scores, for example: – daily internet access (MUSCULO-PER, p<.004) – daily use of the computer (MUSCULO-PER, p<.010) – daily use of tablet for social networking (MUSCULO-PER, p<.025) – daily use of tablet for web conferencing (MUSCULO-PER, p<.028) – gaming more than once per day (INFECTIO, p<.04) One significant positive association – daily usage of the tablet’s phone (HAEMIMMU, p<.032)

12. Students reported tablets as important academically, professionally and personally %n Tablets were important for academic work95.221 Tablets were important for life outside of the university10021 Tablet ownership influenced peers to procure a computing device 95.021

13. *33 months post intervention *

14. Outcomes and Impact Portable access to resources “It helped me during my ward rounds and bedside learning and in preparation of my presentations using android apps for medicine like medscape, skyscape and others” “I have some medicals apps that I do use which increase my performance [because I check in the] field what I don’t know.” “I have been able to know…different medical issues and discoveries in medicine.” “Academically I am able to access online materials”

15. Outcomes and Impact Key literacy development “I [now] read often per day.” “As far as it is easier to carry it [everywhere] it indirectly increase[s] my study reading.” “The tablet has improved my skills on electronic devices” “…improving skills [on] other computing devices and in communication.”

16. Outcomes and Impact Community networking and communication “I have been able to access my emails without going to internet cafes.” “The tablet helps me in communication either by sms, phone or by social networks.” It has also impacted the way I use social networks. …communication with my colleagues has also become easy. …helps me to socialize with the world, interact with my friends.

17. Main Problems Reported Issues of “isolation” and “dependence” Limited internet access Device malfunction (crashes) Support wishes Free, high-bandwidth connectivity Access to protected sites and resources (e.g. Hinari) App and online resource orientation Training on very specific apps (e.g. UpToDate) or tasks (presentation creation)

18. Lessons Digital literacy and self-efficacy may be positively impact with tablets in Rwanda, while decreasing overall usage of computers Learner agency, resource access and sense of community may be positively impacted with tablets in Rwanda, but internet access is limiting

19. Lessons In context, summative semester exam scores are probably not a satisfactory measure of impact (on their own) – Exams may change semester to semester – Exams may not measure what we actually want to know In general, it is helpful to measure usage and impact broadly in technologically marginalized contexts – We may miss key issues by focusing on narrow measures, or by not measuring. – There is evidence that extremely constructionist interventions which use technology may sometimes disrupt technologically marginalized learners See Warschauer and Ames (2010) look at the impact of OLPC See Reardon (2011) and Baily & Dynarsky (2011) about the widening education and achievement gaps in the US

20. Lessons Lightweight, mixed methods monitoring and evaluation strategies which are adaptable and active over longer periods of are helpful and feasible: Monitor small indicator sets over time for implementation guidance – SMS: Textit (textit.in) RapidPro (rapidpro.io) – Email/web: surveymonkey, polleverywhere Qualitative open-ended questions and focus groups – Barriers/solutions/support gaps, – Relevant resources – Innovative user strategies in context – Development of CoP and participation – Wikisurveys: allourideas.org Mobile device management (MDM), Web and Learning Analytics

21. Thanks! Adam Papendieck apapendieck@gmail.com @apapendieck

22. Health Systems Transformation Faculty development Educational Programs Applied Research: policy, operations and evaluation Administrative Systems Faculty and student extension/outreach Higher Ed. PartnershipsHealth systems change Improved program policies, program design and implementation Sustainable production of health workforce needs in country Systems Change

23. Student graduation growth in Rwanda Source: Rwanda Higher Education Council (2011)

24. Research Context Extremely constructionist tech interventions might further disrupt technologically marginalized – See Warschauer and Ames (2010) look at the impact of OLPC – See Reardon (2011) and Baily & Dynarsky (2011) about the widening education and achievement gaps in the US