Completing the post KAP survey

Purpose of post campaign KAP survey Assess the impact & success of your campaign and determine next steps

How do we achieve that? By measuring SMART objective attainment and exposure to campaign activities

What data are we looking for ? Measure for each step of the results chain for each TA = SMART Objective Questions Exposure Comparability of data Audience segmentation

Further objectives? Collect baseline data Understand the preferred media sources Identify trusted sources Ground truth assumptions Identify Barriers/Benefits to BC.

Objective By the end of the day you will have: finalized your post campaign KAP survey questionnaire, sampling methodology & plan. revised and finalized your SMART objectives & result chains.

The steps

Step 1: Revise your sampling methodology and create a survey plan. Step 2: Check your SMART objectives and mark them in your survey Step 3: Revise your campaign survey in word Step 4: Hide the questions in Survey-Pro Step 5 & 6: Implement your survey and carry out data entry

Schedule for the day 8-9 amIntroduction to shortening KAP survey (AB) 9-10 amStep 1: Revise your sampling methodology and create a survey plan. Check in with PPM before finalizing 10am -12 pm Step 2: Check your SMART objectives and mark them in your survey. Check in with PPM before finalizing 1 -2 pmStep 3: Revise your campaign survey in word using track changes pmCheck deletions with PPM pmStep 4: Hide the questions in Survey-Pro that you deleted from the word version pmRefresher on training enumerators, data entry, top tips and Q/A (in preparation for Step 5 & 6)

Step 1: Revise your sampling methodology and create a survey plan. Decide which groups you will sample for the post campaign KAP survey and revise your sampling methodology accordingly. Create a realistic post-campaign survey plan (to be finalized once you have revised your survey and tested survey length).

Decide which groups you will sample for the post campaign KAP survey and revise your sampling methodology accordingly Keep primary audiences (fishers & community) Other groups are optional For each TA sampling methodology should be exactly the same as for pre-KAP

Sampling methodology template A.Define geographic scope of survey and include a map showing area included in survey B.Sample size calculation & sampling methodology C.Survey plan

B. Sample size Name of TA Total n of TA (and who it includes with breakdown Sample size (specify CL and CI) Final sample size achieved

B. Sampling methodology N of TA in each locality N of samples per TA per location (stratified sampling) Method of sampling: - for pre – KAP: amend to reflect reality - for post KAP – same as amended for Pre If there are any differences between pre-and post please take note of these in the table

A. Define geographic scope of survey and include a map showing area included in survey

C. Create a realistic post-campaign survey plan (to be finalized once you have revised your survey and tested survey length).

Step 2: Check your SMART objectives and mark them in your survey CHECK THAT: 1.Smart objectives are aligned with your results chain 2.Smart objectives enable you to measure the success of your campaign 3.There are 1-2 obj. for each section of the ToC, for primary TA - link to your ideas for BC measurement from yesterday 4.Data in the smart objectives matches your pre-campaign results and the answer options of those results 5.Smart objectives targets are realistic (22K, 13A, 28IC, 14BC 6. Question numbering in the monitoring plan matches final survey

Whats the issues? Result chain (BR): Arbor fishers and gleaners actively participate in the strict enforcement of Arbor MPA NTZ regulations by participating in MPA enforcement activities. Smart Objective: The % of Arbor fishers and gleaners that are able to give at least one benefit of establishing the Arbor/Granada Marine Sanctuaries will increase from 16% measured in February 2011 to 26% measured in May 2012 (an increase of 10pp, Q5 in KAP survey)

Correction Result chain (BR): Arbor fishers and gleaners actively participate in the strict enforcement of Arbor MPA NTZ regulations by participating in MPA enforcement activities. Smart Objective: The % of Arbor fishers and gleaners that are able to give at least one benefit of establishing the Arbor/Granada Marine Sanctuaries Management Committee will increase from 16% measured in February 2011 to 26% measured in May 2012 (an increase of 10pp, Q5 in KAP survey)

Alternative Results chain (BR) There is a functional 24/7 MPA patrolling and guarding in place at Arbor and Grenada NTZ SMART objective The % of Arbor fishers and gleaners who agree that there Arbor and Granada MPAs have a functional 24/7 guarding and patrolling system in place ("Yes, in Arbor but not in Granada" AND "Yes, in both Arbor and Granada") will increase from 23 measured in February 2011 to 33 measured in June 2012 (an increase of 10pp, Q29 in KAP survey)

WHATS WRONG HERE? 44% 36%

Step 3: Revise your campaign survey in word Make sure the survey is your latest version Make revisions in English version (cross check with vernacular) Use track changes

Guiding principles for shortening surveys All questions should remain exactly the same (even if they had a mistake) Questions and answer options should remain in the same order Reduce survey to 30 minutes – max 40

Should be retained: 1.Socio-economic data - Section 2 2.Questions you used during analysis for cross tabulations and filters (make sure you check which ones these were. i.e. sampling methodology, which category best describes you) 3.Compulsory cohort questions, selection to be retained (see provided list in Appendix 2) 4.Climate change questions 5.Questions needed for your SMART objective

Can be removed : 1.Trusted Sources of Information & Media Access/Exposure- Section 3 2.Flagship sp question - Section 11 3.Any questions that you no longer deem useful, including cohort questions, provided they are not part of the categories to be retained

To be added: 1.A question asking respondents whether they have taken this survey before answer options to the exposure question I am going to ask you about a number of ways in which you may or may not have seen or heard about the XXX Sanctuary. For each method, I would like you to tell me whether you remember seeing or hearing that source in the past 6 months. - Section 10 Answer options should relate specifically to campaign events, activities or materials that you implemented over the last 6 month. Expl: boat awnings, battle of the bands, or MPA blessing retain at least two dummies as a measure of false negative

Finally please make sure Under survey period there is a post campaign survey answer option – Section 1

Step 4: Hide the questions in Survey-Pro Open survey pro Go to the questionnaire tab Double click on the question you want to hide Click on the advanced options tab On the right select Hide for all answer options The question will still appear in the survey but all answer options will be hidden For grid questions: The process is the same as above but at n4 above click on rows.

Step 5 : Implement your survey

Choosing & Training enumerators Enumerators neutral in the community Should be committed to survey so there is not a high turnover Should receive a day training & practice Have clear instructions in the survey for enumerators

Training enumerators agenda Intro to campaign => Motivation Ethos / Neutral approachable attitude Process (one on one; dont share paper but interview)

Run through questions and instructions one by one (translated version) Keep tally of respondents who dont want to be interviewed Record any changes in sampling methodology Have feedback system Training enumerators - agenda

Go over survey plan Make sure enumerators understand sampling methodology random sampling – (map + dice to select street?) stratified sampling – (create simplified version) Provide additional materials needed (MAP for survey question, survey plan with MPAs of area, Community Based Monitoring System (CBMS) map from MPDO; ID) Do a practice run & debrief Training enumerators - agenda

(55) Please name 3 creatures that live in the sea that you are particularly fond or proud of. (do not read out options but tick box as appropriate and/or fill in the blank for any additional answers) FLAGSHIP SP QUESTION - 1st OPTION [ ] grouper "lapu-lapu"[ ] giant clam[ ] giant sea fan[ ] XXX ADD OPTIONS AS APPROPRIATE[ ] XXX ADD OPTIONS AS APPROPRIATE[ ] Other ________________[ ] N/A Instructions for enumerators

Top Tip Laminate survey questionnaire with hole and ring binder Create answer sheet => Instructions for enumerators in both documents

Do a pilot survey 3-5 per enumerator (test on people random sample in least important area first) Do a debrief from pilot survey Training enumerators

Agree on n of surveys per day and target Have organizer within group Safety protocol At a minimum take part in first 2 days of surveying Do data entry of pilot survey with encoders!

Step 6: carry out data entry

KAP Survey TOP TIPS Encoding Dry run Change the structure of the open ended questions before encoding Encoders who are willing to work evening/night shifts. Dont try and do all the encoding yourself: get some help! Be strategic about the order in which you encode (and also survey) the target audiences start by encoding fishers and the local community (TA1 &TA2) Write the corresponding form number in Survey pro on the paper survey

Data entry Bottleneck! Start early Save every form System for keeping paper copy => BACK UP every day!

Good luck!

Diskarte