Innovations in Metrics of Learning LINGOS meeting London Meeting 18 th September 2009 Anne Kristine Arbon (former Save the Children UK) Arbon Consulting

Definition of Learning Learning is the process by which individuals, teams, organisations and groups of organisations create, transfer and use knowledge in order to achieve positive change and realise their goals.* *Definition: Learning by Field Level Workers, ALNAP Review of Humanitarian Action in 2003, Publication date: 2004 ( by Field Level Workers

The Difference between Data, Information and Knowledge Data discrete, unorganised facts. Information data that is organised into groups or categories which can alter the way a person perceives something. Knowledge familiarity, awareness or understanding gained through experience or study. ‘Because knowledge is intuitive it is difficult to structure, can be hard to capture on machine and is a challenge to transfer.’ Definition: IO Council (2001).

How does learning happen? Information Knowledge

Why do we measure? ROI (Return on Investment) Mission critical What kind of data do we use to measure impact of our learning? - How do you know the data is relevant? - Who says so?

Deciding on measurements Your trip here: Process: length of travel, time spent on travel, mode of travel, comfort of travel, cost of travel Result: arrival on time, harassed, happy, tired, annoyed (Length of travel – time spent on travel)* cost of travel Proximity of other commuters =

What should we measure? Process OR Results OR Both Or? Kirkpatrick 4 levels of evaluation

Example of ROI % ROI = (benefits / costs) x 100 payback period = costs / monthly benefits Duration of training 33 hrs Estimated student numbers750 Period over which benefits are calculated12 months Costs Design and development£40,930 Promotion£4,744 Administration£12,713 Faculty£86,250 Materials£15,000 Facilities£40,500 Students£553,156 Evaluation£872 Total cost£754,165 Benefits Labour savings£241,071 Productivity increases£675,000 Other cost savings£161,250 Other income generation£0 Total benefits£1,077,321 Return on investment143% Payback period8 months

What information do we have available?

Save the Children Metrics for global Core Learning: No. of staff uploaded No. of people who are in progress/completed courses Surveymonkey results of e-modules (incl. qualitative data) No. of face to face core learning workshops held No. of participants on core learning workshops

Where does the money go? "...international staff accessed approximately 10 times more explicit knowledge assets from their organisations than the national staff. International staff also attended co-coordinating structure meetings at approximately 10 times the rate of national staff..." ALNAP LD P Senior mgt dev Managemen t developmen t Core learning Total investment since 2004 from HQ No of staff Training programmes + 3 % of country programme salary budget £ £ £ £

How can we use metrics to change where the money goes? What metrics will your organisation use to measure learning solutions?

Thank you for listening!