Insert organisational logo here on master slide TECHNIQUES

Insert organisational logo here on master slide REMEMBER All projects should have –specific aims, –declared outcomes –appropriate methods –sufficient resources –sufficient resources (money, people and time) –should be evaluated at the end point.

Insert organisational logo here on master slide What techniques do you use? Resources and time will often dictate what you can do. Quantitative = How many people agree/disagree with the changes Qualitative = Why they agree/disagree

Insert organisational logo here on master slide TECHNIQUES Qualitative and Quantitative Not either/or but mix and match There is a need for Qualitative and Quantitative to give a complete picture (This is often a luxury) Whatever technique used beware of BIAS. Trade off between bias, convenience and pragmatism

Insert organisational logo here on master slide Many problems will be avoided if a PILOT is done first A pilot is a small scale replica of the main survey All parts of the process must be piloted Be prepared to make the changes Re-pilot if necessary

Insert organisational logo here on master slide Be brave, abandon rather than waste everyone's time

Insert organisational logo here on master slide What techniques could we use? (1) Open Public Consultation –Meetings –Websites –The Media –Open Days, Road shows Citizens’ Panel Public and/or Patient Forum –Standing Panels/Health Panels Patient/Lay representation on specific groups –Reference group –Liaison Group –Management Group –User Group Focus Groups

Insert organisational logo here on master slide What techniques could we use? (2) Surveys –Interview –Questionnaire Patient stories/ Patient diaries Process mapping Observations of care Compliments/ Complaints Staff feedback Audit Community Development Initiatives Participatory/Rapid Appraisal

Insert organisational logo here on master slide Who will collate the responses? be responsible for quality control, coding and data entry? undertake the analysis and write the report? develop any recommendations and action plan? ensure the changes happen? disseminate the results/feedback?

Insert organisational logo here on master slide WHO DO WE ASK? Everyone –Patients/Users –“Public” Community Health Council Voluntary Sector

Insert organisational logo here on master slide WHO DO WE ASK? A sample Representatives All patients “Nice” patients “Stroppy” patients Colleagues Friends –ours –patients’ Relatives –ours –patients ’

Insert organisational logo here on master slide HOW? Identify whole population of interest What characteristics/variables might affect findings (age, gender etc)? Try to identify a true random sample - convenience samples usually have bias (pub closing time in A&E etc)

Insert organisational logo here on master slide DO NOT FORGET ETHICAL APPROVAL DATA PROTECTION INFORMED CONSENT

Insert organisational logo here on master slide PATIENT SATISFACTION SURVEYS HAVE A BAD NAME! Why? No clear purpose behind many in the past Superficial No changes made BUT they can be very useful and informative

Insert organisational logo here on master slide Benefits of Questionnaire Surveys Cheap Can be done by one person but can separate design, data handling etc Repeatable Cover wide geographical area Larger sample Observer bias minimal (take care with question design)

Insert organisational logo here on master slide Drawbacks with Questionnaire Surveys Low response rate - reminders may be required Time lag between questionnaires out and returns Sample bias, literate, language etc No clarification possible, and the qualitative bit often missing Difficult to inform respondents what the outcome was, need to “tell the world” Data collection and interpretation may be difficult

Insert organisational logo here on master slide Benefits of interviews Better response rates In depth – qualitative Flexible, can follow up leads Questions can be clarified Fewer unanswered questions Interview questions can be refined during the process if unexpected findings arise or focus on specific problems if necessary Tape with permission ~1 hour

Insert organisational logo here on master slide Drawbacks of interviews Social and ethnic characteristics of interviewer may affect responses Expensive, transcribing time ~8:1 Coding difficult Right answers May need interpreters Need to make appointments

Insert organisational logo here on master slide Benefits of group Short time scale Can work with only one skilled person Can repeat with cohort, before, during and after service changes Can include those with low literacy skills Stimulates debate Good at identifying cultural values Gets those with nothing to say involved Good back up qualitative for quantitative

Insert organisational logo here on master slide Drawbacks of groups Not representative, volunteers are self- selecting and may want to “hijack” the group Can change ideas during discussion when more information available Difficult if transport an issue! Transcription and coding can take a long time

Insert organisational logo here on master slide Benefits of Public meetings Everyone and anyone invited to attend All attendees invited to contribute Potential for large numbers of people to become involved

Insert organisational logo here on master slide Drawbacks for Public Meetings Meeting “hijack” Many people unable/unwilling to speak out in public Many staff often required to man event Difficult to give sufficient background information in the time available Poorly attended

Insert organisational logo here on master slide Whatever technique used, the questions posed must be relevant unambiguous give some simple options in appropriate language

Insert organisational logo here on master slide Questions must not be offensive biased leading full of assumptions

Insert organisational logo here on master slide EVALUATION Comprehensive report available –Results presented in understandable format All stages described, aim, method etc Method –Justified –Repeatable Outcome measures specified Discussion of biases etc Context of other work given Recommendations follow from results

Insert organisational logo here on master slide WHAT COULD BE IMPROVED ABOUT THE METHODS? What facilitated/inhibited involvement? What influenced implementation of recommendations? What benefits/drawbacks were there for those who became involved? Who else could/should have been involved to increase information or changes made?

Insert organisational logo here on master slide INFORMATION USED EFFECTIVELY? What changes were made? Were the respondents consulted about the changes? Were there conflicts of interest? Dialogue continued between professionals and public Could the results be generalised?