1. Strengthening KM Measurement and Process
Saori Ohkubo & Tara Sullivan
Knowledge for Health (K4Health) Project
GHKC Meeting
October 22, 2015
So for the next 45 minutes or so, we will look at the issues surrounding the future of monitoring and evaluating knowledge management. While one of the focus areas will be the session title, “improving and measuring collaboration” we will be also covering other related issues, in particular, different components in building KM capacity and nurturing KM culture.

2. GHKC Achievement to Date
Guide to Monitoring and Evaluating Knowledge Management in Global Health Programs
KM for Global Health Logic Model
GHKC M&E Task Team
I want to start with recognizing the accomplishment that this group, GHKC, have made in terms of advancing M&E of KM for global health.
KM M&E guide
It includes an unique KM logic model, and 42 common KM indicators mapped to key elements of the model, as presented by Tara.
Each indicator is described in detail, including its definition, data requirement, data source, purpose and issues, as well as illustrative examples where applicable
In covers emerging themes such as social media, interactive usability assessment, web analytics, and community of practice, presenting proven or best practices, and making recommendations for appropriate metrics to consider.
It describes a variety of data collection techniques and methods
It includes useful and practical tools and sample instruments

3. Proposed Next Steps Reconvene M&E Task Team
Work together on KM M&E tool(s)
KM capacity assessment tool
Continue our dialogue on emerging topics such as
Using stronger research designs
Measuring the “social” aspects of KM

4. Assessing KM Capacity and Culture
K4Health Knowledge Management Capacity Assessment Tool in Development
Center piece of the processes in the KM logic model is “strengthening of culture and capacity.”

5. Knowledge is not static nor permanent. Knowledge grows.
Knowledge management needs to be adaptive and flexible to continuing changes within the individuals, organization, and surrounding environment.
How to foster collaboration and become a learning organization?

6. K4Health KM CAT: Background
It draws from and builds upon various resources (esp. APQC’s KM CAT and MSH’s PROGRES)
It is a participatory self-assessment tool
It aims to measure KM maturity and competency of an individual organization and/or a group of organizations
It includes both quantitative and qualitative data collection approaches
Its findings should be used to identify concrete action items to advance through the capacity continuum
Resources:
Capacity building/strengthening assessment tools in the field of social and behavior change communication and KM developed by CCP
Organizational capacity assessment guidance and tools developed by other international development agencies (e.g., World Bank, PACT)
KM capacity assessment guidance and tools developed by private sector companies (e.g., APQC)
Literature review of KM maturity models
Other relevant journal articles
KM CAT is a participatory self-assessment tool to identify existing skills and assets as well as areas for improvement to manage KM
It aims to measure KM maturity and competency of an individual organizations and/or a group of organization (e.g., professional networks, technical working groups)
It includes both quantitative and qualitative data collection

7. Five Stages: Capacity Continuum
Not present
Not started
Stage 2:
In-development
Limited
On ad-hoc basis
Stage 3:
Standardized
Formalized
Not applied
Stage 4:
Applied by some staff/unit
Stage 5:
Applied completely
Improved continuously
KM function, procedure, activities, etc. are…
Changes in KM capacity and culture happen in continuum, moving from one stage to the next, from lower to higher stages.
Stage 3 is the most important milestone in an organization's journey toward KM maturity/competency because it denotes standardization.
Standardizing and implementing KM approaches and processes will build the foundation for a knowledge-sharing culture.
(Adapted from MSH’ PROGRES and APQC’s KM CAT)

8. Five Ps: Core Domains
People covers 1) resources/human capital required for KM, 2) leadership, which is the organization’s senior management support to KM, and 3) organizational culture supporting knowledge sharing and networking.
Process refers to 1) KM strategy that is aligned with the broader mission of the organization, 2) knowledge flow/cycle of assessing, capturing, generating, adapting, and sharing knowledge within organization, and 3) measurement such as M&E system and indicators.
Platform includes the organization’s use of 1) KM systems, 2) KM approaches, and 3) information technologies.
Partnership refers to the organization’s involvement in collaborating with key stakeholders and partners for 1) knowledge exchange, 2) coordination, and 3) networking purposes.
Problem solving is a vehicle for connecting knowledge and performance and covers skills such as 1) knowledge seeking, 2) identification of new ideas and problems, and 3) critical thinking.

9. Resource (staff and training)
Leadership (leaders/managers, coordinating body)
Culture (support to networking and sharing)
People covers 1) resources/human capital required for KM, 2) leadership, which is the organization’s senior management support to KM, and 3) organizational culture supporting knowledge sharing and networking.

10. Strategy (common vision, objectives, budgets)
Knowledge flow (knowledge cycle elements)
Measurement (indicators, systems, data use)
Process refers to 1) KM strategy that is aligned with the broader mission of the organization, 2) knowledge flow/cycle of assessing, capturing, generating, adapting, and sharing knowledge within organization, and 3) measurement such as M&E system and indicators.

11. KM system (repository/sharing, content management)
KM approach (tools and techniques)
Information technology (IT team, use of IT)
Platform includes the organization’s use of 1) KM systems, 2) KM approaches, and 3) information technologies.

12. External knowledge (gathering and promotion)
Coordination (partnership and structure)
Networks (participation to professional networking)
Partnership refers to the organization’s involvement in collaborating with key stakeholders and partners for 1) knowledge exchange, 2) coordination, and 3) networking purposes.

13. Knowledge seeking (initiative, locating knowledge)
Identification (of new ideas and problems)
Critical thinking (representation and group solution)
Problem solving is a vehicle for connecting knowledge and performance and covers skills such as 1) knowledge seeking, 2) identification of new ideas and problems, and 3) critical thinking.

14. Glance at KM CAT in Action
K4Health KM CAT in Google Sheet
Example of scoring sheet
Example of reporting

15. Discussion Questions
Thinking about your work environment/context, when, how, and with whom do you think you might use a tool like the KM CAT with? How does it support your work? How does it benefit your audiences?
Do main domains (5Ps) resonate with you? Is there anything you would suggest changing, adding, or deleting?

16. Using Stronger Research Designs Measuring the “Social” Aspects of KM

17. Case Study for Research Design
Bangladesh Knowledge Management Initiative (BKMI)
eHealth pilot in 2 divisions (Sylhet and Chittagong) with eToolkit and eLearning videos for field workers to use on netbook computers when counseling mothers
Baseline Survey
April 2013
Field workers
N= 304
Mothers with children < 2 years
N = 674
Endline Survey
September
2013
N = 675

18. Quasi-experimental Pre-Post Design
Tier 1 Field workers Tier 2 Mothers with < 2 yrs child O
X O
April 2013
Sept 2013 O
X O
April 2013
Sept 2013

19. BKMI-II Research Design
Cluster Randomized Control Trial with 3 arms:
Randomization of districts into either Arm1, Arm 2 or Arm 3
Arm 1: eToolkit + IPCC training (2 districts)
Arm 2: eToolkit (2 districts)
Arm 3: Control (2 districts)
Baseline Survey
January 2015
All HAs (field workers) in 6 study districts
Mothers with children < 2 years
N = 800 per district
Endline Survey
January 2016
All HAs (field workers) in the 6 study districts

20. Malawi Net-map

21. 2010 Baseline/Endline Net-Map Information Flows with District Level Stakeholders in Nkotakota

22. 2010 Baseline/Endline Net-Map Information Flows with District Level Stakeholders in Salima

23. Social Knowledge Management
Social benefit
Social systems
Social networks
Social learning
Social capital
Social software/media

24. Resources
Guide to Monitoring and Evaluating Knowledge Management in Global Health Programs
Leveraging Knowledge Management for Public Health
Conceptualizing a New Knowledge Management Logic Model for Global Health: A Case-Study Approach
Taking Knowledge for Health the Extra Mile: Participatory Evaluation of a Mobile Phone Intervention for Community Health Workers in Malawi
The Intersection of Knowledge Management and Health Systems Strengthening: Implication from the Malawi Knowledge Management for Health Demonstration Project

25. Thank you and stay tuned for a follow-up message about the KM M&E TT via GHKC email