Using Outcome Mapping as a Research Methodology for Information Communication Technology Projects Chris Morris African Advanced Institute for Information & Communications Technology (AAIICT)

Comparative study of “first mile” and “first inch” technology in different low density contexts. Project funded by the IDRC

Mozambique Angola South Africa Zimbabwe FMFI Projects & Partners Angonet (Huambo) Wireless Mesh (Peebles Valley) CSIR Telehealth (Eastern Cape) UWC MuTi (Eastern Cape) CSIR HLT (Openphone) Translate.org (HTML Pub) Power Line Comms (Rooiwal) SchoolNet Mozambique (Inhambane) MICTI (Maputo) UCM (Pemba) ISPU (Quelimane) Wireless (Harare)

Objectives The objectives of this project are:  To develop innovative ICT technologies and to implement “first mile” solutions  To develop business models for first mile solutions  To research behaviour in the use of ICTs – how the use of ICTs has changed community life  Scalability and replicability of technologies – the viability of rolling out the solution Influence on policy and regulation – demonstration of project benefits to the regulator  To publish a reference book for “first mile” and “first inch” implementation in rural Africa.

Research Question? What is the sector you want to contribute to through your project? How does it link to the overall vision of the First Mile/First Inch Network? What is it that you want to make known through your project? (What is the research question?) Why? Who are you going to influence through your project? What kinds of influence are you intending to have on your boundary partners’ behaviours? What are the main methods by which you are going to answer your research question? Why did you select these methods? (relates to issue of validity) What are the main strategies you are going to use to influence your boundary partners? Why did you select these strategies? (relates to issue of relevance) How would you propose to monitor and report on the project process as it unfolds?

Choice? research methodology that was simple to apply robust in being able to provide an implementation framework but flexible enough to adapt we needed to be participatory with our project partners to understand the development process and learn more about human behaviour and how that influences technology choice we are a learning organisation, we wanted to look at the change taking place, not simply make a claim to it. a tool that helped us document the journey & understand the processes - what worked and what did not work and why one that would develop a space for social learning and creative dialogue

Choice contd. ? we needed to be able to reflect and review our plans and make changes when necessary. at the end of the project we needed to know where we have been and what were the lessons learned. we needed to know how to improve next time. to be accountable to our stakeholders and donors. we needed to undertake comparative research across projects, across countries. we needed a dissemination strategy - identify Primary Intended Users and Primary Intended Uses of research results.

Why Outcome Mapping? Outcome Mapping methodology for planning, monitoring, and evaluation in order to document, learn, and report results  Development is accomplished by, and for, people  Focuses on outcomes as behavioural change  Not impact but contributions to outcomes  Boundary partners control change

Outcomes Mapping  OM provides a method for development programmes to plan for and assess the capacities that they are helping to build in the people, groups, and organizations that will ultimately be responsible for improving the well-being of their communities.  Outcomes are defined as changes in the behaviour, relationships, activities, or actions of the people, groups, and organizations with whom a program works directly  Boundary partners are those individuals, groups, and organizations with whom the program interacts directly and with whom the program anticipates opportunities for influence

Outcome Mapping…Cont. Outcome Mapping asks the following questions :  Why? (What is the vision to which the program wants to contribute?);  Who? (Who are the program's boundary partners?);  What? (What are the changes that are being sought?); and  How? (How will the program contribute to the change process?).

Why? VISION  This inspired ambitious action and commitment, - but was not an yardstick by which the project would be measured

Who? - Boundary Partner Map

Outcome Challenges Boundary partner – Clinic sister The project intends to see the clinic sister empowered by the use of the technology and sharing her knowledge with other clinic sisters in the area and seeking advice from specialist. The clinic sister will build a trust relationship with the doctors and be recognised by other health officials for inputs into decision-making. The sister is able to clearly articulate a vision of telehealth activities and goals that is relevant to her community needs. She acts as a champion for the roll-out of telehealth in the province and motivates others. Boundary partner – hospital doctor The project intends to see the doctors supporting and participating in the use of the technology. The doctors will join the online community of health specialists and share case studies and lessons learnt. The doctor will build a trust relationship with the clinic sister and help build her skills and knowledge.The doctor will access health information on the internet and continue with his professional training online. The doctor will use the internet to keep in professional and social contact with colleagues and friends around the world thereby reducing some of the feelings of isolation experienced from being in a deep rural location. Boundary partner – East London hospital specialist The project intends to see the dermatologist based at the East London hospital responding to the clinic sister’s s and providing advice. on a timely basis. One would also like to see a proactive stance from the doctor in terms of devolving first line diagnostic decisions to the clinic level, based on the learning experience of clear-cut cases. The dermatologist will build up case studies and share and promote the technology at conferences and medical events. Boundary partner – Provincial Department of Health The project intends to see Dr Rajeev become an enthusiastic supporter for the telehealth technology. He will influence the Department to adopt the Tsilitwa telehealth project and seek to replicate it throughout the Eastern Cape Province..The rural telehealth solution will be included in the Department’s strategy and budget. The Department will develop training modules for the clinic sisters and provide ongoing support. Dr Rajeev will become a champion for rural connectivity and inspire other provinces and the National Department of Health to replicate the technology.

What? - Progress markers Outcomes have been assigned to each boundary partner & progress markers are used to monitor behavioural changes of boundary partners in achieving the desired outcome. The Progress Markers for the Outcome Challenges are categorised according to the current status where: Expect to see =Behaviours the project expects to see Like to see=Behaviours the project would like to see Love to see =The behaviours associated with achieving the best outcome

Progress Marker: Clinic Sister Clinic Sister Expect to SeeLike to SeeLove to See Clinic sister not afraid to use the system and feels like it is a benefit Sister becoming proficient in use of the technology and suggests improvements Sisters becomes a champion for rural telehealth, she develops self- confidence - talking at conferences,, motivating others, advocacy and training Sister feels confident to communicates with the Nessie Knight hospital doctor using the technology Some communication with other provincial doctors and specialists (Umtata, East London) Sister builds up experience and a body of case studies, reading and ongoing skills development on the internet (Consulting medical documents for improved health care) sharing new knowledge with other health workers The sister’s experience grows and she performs simple diagnoses of patients with some support by the doctor The clinic sister feels confident and empowered showing off and training other healthworkers. Sister gives inputs into provincial telehealth discussions and influences telehealth plans The community are not afraid to be diagnosed through the use of the technology Sister accesses the internet for information and uses it for ongoing skills development Sister articulates a vision of telehealth to the community

How? - Strategy Map StrategyCausal I-1Persuasive I-2Supportive I-3 Strategies and activities aimed at a specific individual or group Supply ICT equipment Deliver training to the sister Make sure the system is reliable Provide technical support from doctors requesting cases Regular contact with sister from CSIR Brief the DoH (Hospital super, matron, DM, Dr Rajeev, MEC with regard to the system to ensure ongoing support for the sister E-1E-2E-3 Strategies and activities aimed at individual’s or group’s environment Include Telehealth in sisters KRAs Telehealth ICT policy Frequent visitors to see telehealth UNDP country visits Promote the project through the media Collaboration with other organisations

OM 3 STAGES Intentional Design Why? Who? What? How? Outcome & Performance Monitoring Outcome journals Strategy journals Performance journal Evaluation Planning

Organisational Practices Prospecting for new ideas Opportunities and resources Seeking feedback from key informants Obtaining support from your next highest power Assessing and redesigning products, services, systems and procedures Checking on those already served to add value Sharing your best wisdom in the world Experimenting to remain innovative Engaging in organisational reflection

Evaluation Assess contribution to development outcomes Understand linkages between behavioural changes and strategies User uptake, benefits of ICTs Influence on regulatory policy Documentation of lessons learned Understand intended uses and intended users By using OM as a framework it allows comparison across cases, across countries

Dissemination Strategy PI Users PI Uses Influence on regulatory policy : Department of Communication Create license exemption for use of wifi in research Create license exmption for use of wifi for non-commercial use e.g. health & education Dissemination strategy – inner circle, LINK, DoC, NGOs, University Influence on health policy : Department of Health Eastern Cape Support for the use of wifi for telehealth in rural clinics Demonstrate value of eHealth training module Focus group key DoH decision makers,Provincial Head, Supply Chain Manager

Conclusion OM as a research tool enabled us to answer the research questions – what is it that you want to make known that was not known before? The research produced primary data in discovering something new. OM provided the route map and data gathering tools to produce evidence based results. The evaluation of these research results enabled us to answer the research question user uptake, influence on policy OM provided a methodology for the comparative analysis between projects, between countries OM provided the framework for planning, documenting the journey, and capturing the lessons learned during the life of the project. It helped the project team document “where it had been”, understand the development process, social learning & creative dialogue OM enabled the project team to review progress and adjust strategies when necessary recognising development does not have fixed goals but is a moving target. OM provides a framework for understanding who are the boundary partners with whom the project seeks to influence and can influence, what are the uses and who are the intended users of the research results.

Thank You Chris Morris ICTs for Development Tel: +27 (12) Cell: URL: URL: