Victim Surveys Part 1 Crime, Justice & Security Statistics This part is about what can be done with the results of the Victim Surveys and about what Surveys exist in various countries, what the international guidelines are and how telephone surveys can be used. Produced in Collaboration between World Bank Institute and the Development Data Group (DECDG)

Victim Surveys as an alternative source of data Asking people their experience of crime can give a fuller picture than police statistics This is usually done by conducting a victim survey among a sample of households Such a survey needs careful planning to be cost-effective It will need to be carried out by an organisation that includes well qualified statisticians This Module looks at the main considerations when planning a victim survey Victim surveys are an alternative source of data on crime to those collected by the police (see Module 3) Figures obtained from such surveys usually give a fuller picture than that obtained by the police, because not all crimes are reported to the police. Victim surveys can be carried out among many different types of victims: a survey of households is the most common, but surveys of tourists or of commercial properties are also sometimes carried out. Any victim survey needs careful planning: it is not possible to ask all households about their experience of crime as the cost would be far too great: so some form of sampling would be needed. The choosing of the sample, the design of the questionnaire and the carrying out of the survey requires a considerable degree of competence in statistics: this means a victim survey has to be carried out by an organisation that includes well qualified statisticians experienced in carrying out surveys of households. This Module looks at the main considerations when planning a victim survey.

Who should carry out a victim survey? The most likely agency is the Central Statistics Office, which has the capacity and experience to carry out a survey (e.g. Mauritius) In some countries, however, special organisations have been set up with the similar capacity and experience inside: One of the justice ministries (e.g. UK) local government: e.g. a city authority (e.g. Tanzania) a university or research institute (e.g. South Africa) Examples of different types of victim survey, carried out in a variety of countries will be shown in the slides at the end of this Module. The UN has issued guidelines for conducting victim surveys The choice of the agency to carry out the victim survey will vary from country to country. The most appropriate agency is usually the Central Statistics Office: this will have the capacity and experience to carry out such a survey and will typically carry out household surveys on other topics on a routine basis. Mauritius used the CSO to conduct a victim survey but not all countries do so. In some countries other agencies have the capacity and experience to carry out victim surveys: eg In the UK the justice ministry (Home Office) carries out the British crime Survey: In Tanzania, victim surveys are carried out within local government: typically a large city authority: Finally, in South Africa, victim surveys are carried out by a research institute. Examples of different types of victim survey, carried out in a variety of countries will be shown in the slides at the end of this Module. Around 40 countries are known to have conducted victim surveys, many of these as part of the International crime Victimisation Survey. The UN has issued guidelines on conducting victim surveys and a reference is given to these at the end of the Module.

Main problems of conducting a sample survey How much will the survey cost? What part of the country should be sampled? How large a sample should be chosen? How should this sample be chosen? What should be included in the questionnaire? What classifications should be used? How can the results be used together with the data held by the police? There are several points to consider when thinking about conducting a victim survey in a country. These points are all linked together and the ideal situation may not be possible because of insufficient money, conflicting demands on limited statistical expertise or logistic difficulties in sampling. The main points to consider are: How much will the survey cost? The money available is frequently the determining factor for the type of sample that can be afforded. What part of the country should be sampled? The whole or limited areas How large a sample should be chosen? Small samples will have large sampling errors associated: larger samples will cost more and be more difficult to organise How should this sample be chosen? This will depend on what statistical infrastructure exists What should be included in the questionnaire? A balance has to be achieved between a short focussed questionnaire and a longer one covering more topics What classifications should be used? To what extent can these be identical with other statistical classifications?. How can the results be used together with the data held by the police? This will start with sharing the results with the police and only later move on to discuss whether the two sets of results could be published together.

How much will the survey cost? Costs vary depending on size and complexity If a few questions are added to a survey of households already being conducted (Mauritius 2004, 2007), marginal costs will be low, mainly staff time. If a large stand alone survey is required, especially a full national survey with an extensive questionnaire, the cost can be quite high: e.g. the British Crime Survey costs around US$2m each year. If the survey is included in a wider program to improve justice services, this too can be expensive: e.g. the program associated with the victim survey in Dar Es Salaam in 2004 cost around US$0.5m. However, small scale stand alone victim surveys of some 2,000 households, based on an internationally designed questionnaire, can cost around US$100,000. The costs of victim surveys vary enormously. If a country has not conducted a victim survey before but has a CSO that conducts a household survey, say of basic facilities in a household, the easiest way to start a victim survey is to add questions to this already existing survey. The extra cost would be low, perhaps six months staff time in designing a small number of questions, training the interviewers, and analysing the data. At the other end of the scale if a large stand alone victim survey is required, with an extensive survey of a hundred or more questions involving people’s opinions and actions as well as victimisation, then this can be very expensive. The British Crime Survey costs around 2mUS$ a year, for example. Many of the surveys carried out in developing countries are paid for by donors and are part of a wider programme involving action and improving local justice services. These can also be expensive: for example the Safer Cities Tanzania programme where a survey of 1,000 people was carried out in Dar Es Salaam as part of wider programme costing around 0.5m.US$. A useful compromise between these different costs is to design a stand alone victim survey of no more than 2,000 households, with a questionnaire, based on one designed to e internationally acceptable, that can be answered in around 10-15 minutes, Such as survey could cost around 100,000US$.

What part of the country should be sampled? The ideal solution is to sample throughout the country, perhaps adding a few questions to an existing survey. This will give results that will be true for the country as a whole but, because of sampling error, it may still not be possible to show how victimization varies across the country (e.g. see Mauritius survey). An alternative is to sample households within a capital or another large city. This will give a more focussed result, be easier to conduct, and be more likely to relate to policing policies to reduce crime. The main disadvantage is that it will ignore the rest of the country. The ideal solution is to sample throughout the country, so that rural areas where crime is very different will be combined with urban areas. If an existing survey is already conducted country-wide then adding a few crime questions would be the best way to start. This will give results that will be true for the country as a whole. However, it may not be possible to get accurate results for each separate part of the country. This is because the sample in some geographical areas will be very small, and the victimisation rates for some areas will have very large sampling errors attached to them: eg when you view the results of the Mauritius survey you will see that no victimisation rates are given for sub-regions of Mauritius. An alternative is to sample around 1,000 households within a capital or another large city. This will be cheaper and will give a more focussed result. Because of being conducted in a large city, the sampling is likely to be more straightforward as many registers will exist. The carrying out of the survey will be easier, because of there being less need to travel. Moreover the results for a single city are more likely to be able to be related to policing policies to reduce crime in that city. The main disadvantage of concentrating of a single area is that the rest of country is not represented in the survey.

How large a sample to choose? As sample size is related to cost, the size of the sample will depend on cost to a large extent. However, samples of fewer than 1,000 are not to be recommended, as results will not be accurate. The size of larger samples will depend on what can be afforded. The larger the sample the more accurate the results The British Crime Survey samples 47,000 households nationally but gives very detailed results. If sample sizes of say 1,000-5,000 can be afforded, then there is good reason for concentrating the samples in a small number of large cities to make the work more focussed There are very few efficiencies of scale with surveys so broadly speaking the larger the sample the more it will cost to undertake. Therefore, sample size is closely related to the amount of money available for the survey. However, below a certain size, about 1,000, there is no point in carrying out surveys as the accuracy of the results will be low. This is especially true if you are thinking of analysing your sample by victim characteristics such as age, sex and part of the country. You will very soon get some very small samples: eg for crime against young women and these will have very little validity. Above 1,000 the sample size can be as large as you have the resources to afford. In some countries such as the UK, the size of the sample is very large: the British Crime Survey samples nearly 50,000 households annually. Of course, with this size of sample you get very detailed results with low sample errors. Very few countries can afford this size of sample. If the resources only allow you to sample, let us say, between 1,000 and 5,000 households, then it is good practice to concentrate the sampling in a small number of areas, for example choosing 5 cities with a sample of around 1,000 households in each. This will focus effort and reduce cost.

How should the sample be chosen? If a victim survey is included within an already organized survey, there are no problems with the sampling method If the survey is stand alone, the method will depend on what registers exist: e.g. you could need to sample from the following depending on the local circumstances: National registration system - Electoral register - Local tax register The sample itself should be chosen at random: either one Phase or two A sample is usually taken from some form of register: either the electoral register, a national register of adults if one exists, or a local tax register, if there is a local property tax. The validity of the sample will depend on how well this register is kept up to date and represents the actual position on the ground when the sample is taken. The existence and the accuracy of such registers will vary by country. In the UK the electoral register is used, which tends to get updated annually. There is an advantage of adding questions to an existing survey as the sampling problems have already been solved for that survey. Having chosen the register, the sample is taken at random from that register but the process can be simple or more complicated. In Mauritius for example, the register is divide into a number of Primary Sampling Units, and the first Phase of sampling is choose a random sample of PSUs, with a probability related to the size of the PMU. The second Phase of the sample is to choose a fixed number of households within each PSU. Other countries have a one stage sample where the sample is chosen randomly from the register: however, practical considerations should apply: eg. when conducting a survey from the first time, neighbouring countries that have already conducted a survey should be approached to share good practice: the savings gained are likely to be much more than the air fare.

What should be included in the questionnaire? Again this will depend to a large extent on cost and best results will be obtained from a short survey, lasting no more than 15-30 minutes. As well as victimization other topics could be: Demographic characteristics Circumstances of the victimisation Victim’s experience of law enforcement Victim’s opinions about law enforcement Victim’s fears of crime Crime prevention measures taken by the victim The content of the questionnaire will depend to a large extent on cost. Longer questionnaires tire out the respondent and best results will be obtained from survey that lasts no more than 15-30 minutes. It is essential to record the Demographic characteristics of the household, as it is well known that victimisation varies by income of household, age and gender of respondent. The next decision is whether to widen the survey to ask about more than victimisation by taking advantage of actually being in the respondent’s household. A balance will need to be struck between the length of the questionnaire and the usefulness of the results. A series of questions about victimisation is clearly essential to the victim survey. It is then a choice as to whether detail is sought about the circumstances of the victimisation The respondent could be asked about his experience of law enforcement and his opinion about how well law enforcement works in his area. He could asked about his fear of crime: is he happy about going out at night, for example. Finally he could be asked about any crime prevention measures taken in his house

Topics for basic questionnaire Basic Demographics (age, sex, size and income of household, place) Victimization of two or three offences only experience of burglary, vehicle theft, assault in the last 12 months Circumstances of crime amount stolen, damage caused (personal, vandalism) Was the crime reported to the police? However, some form of compromise between length, cost and usefulness of the results will have to be made. This slide deals with what we have called a Basic questionnaire: this is the items that should be asked in any victim survey: The basic demographics should be recorded: the size of the household, the household income, the geographical area, age and sex of the respondent. In this way the figures from the survey will eventually be able to be grossed up to the total number of households in the area and the variation by different type of household discovered. In a basic survey, it is only necessary to ask about the experience of a small number of offences. Some countries, eg Mauritius, choose to ask households about burglary, vehicle theft, and of assault. A time period over which the experience is sought must be decided. The usual time period is to ask about experience over the last twelve months. Some further information could be sought about the circumstances of any offence reported to the interviewer: eg. how much was stolen in the burglary, how much damage was caused: did the assault result in injury, time off work, a visit to hospital. Finally the victim could be asked whether the offence was reported to the police. This slide refers to a basic questionnaire only and would typically be used if questions were being added to an already existing survey.

Topics for fuller questionnaire As above plus: Victimization of a wider range of offences (see next slide) If crime not reported why not? Victim’s fear of crime in the neighborhood Victim’s confidence in the police: in judges Have you interacted with law enforcement in last 12 months Does your house have door/window locks? Does your car have a garage? A fuller questionnaire would range over a much wider list of topics, be much longer and take a longer time to complete. In addition to the basic list on the last slide, the experience of a fuller list of offences could be looked for. The reason for not reporting the crime to the police could be asked for, giving list of possibilities: e.g. difficulty of getting to the police station no confidence in the police intend to sort the crime out oneself. Respondent’s could be asked their fear of crime in their neighbourhood: e.g. are they afraid to go out after dark: would they walk alone in their neighbourhood to the shops. A further question on the respondent’s confidence in the police and law enforcement generally could be asked. It has been shown that this will vary according to whether the respondent had been in contact with police and courts over the last 12 months, so this experience should also be asked. Finally details of crime prevention could be asked: e.g. does the house have locks on doors and windows: if there is a vehicle is it kept in a garage or on the roadside.

What classifications should be used? Demographic data collected in victim surveys will have the same classifications as CSO collect for other statistics. Offence classifications used in victim surveys should be as close to Police data as possible, with the following differences: Do not ask about murder, sexual crime, illegal drug taking as response may not be accurate Avoid using legal jargon as this could be misunderstood. Answers to other questions should be pre-coded to make analysis more straightforward The classifications used in surveys should be the same or similar to those used in other statistical collections. In order to be able to match victimisation data with other household data, it is necessary to use the same demographic classifications for victim surveys as the CSO will use for other statistical collections. When collecting data on offence type, similar classifications should be used as for the police: however, there are some variations: Respondents should clearly not be asked about experiencing homicide. Clearly they cannot answer for themselves but in nearly all countries homicides are well reported to the police and victimisation is well known. It has been shown that some offences do not get reported well in household surveys, unless careful research is carried out first on the best way to ask the question: e.g. experience of sexual crimes and of illegal drug taking is rarely well reported and conclusions drawn from any answers would tend to be misleading. So respondents should not be asked about their experience of such crimes. Legal jargon should be avoided. Use ordinary words in the vernacular language of the country, rather than the sort of words that tend to be used by the police. e.g.. use expressions like ‘was anything stolen from your house’. In a wider questionnaire, including questions about confidence or crime prevention, answers should be pre-coded rather than open-ended, to make the analysis more straightforward.

Offence classification used in UK One of the largest surveys is carried out in the UK. The slide gives the offence categories which are used by this survey. They tend to be more general than police classifications, although they are broadly in line with them. They tend to use more day to day expressions rather than legal terms: They also remain consistent throughout various repeats of the survey, so that, as the slide shows, comparisons can be across different years

Piloting the survey All questionnaires need to be piloted This means trying out the questionnaire in a small number of households, in ‘real’ conditions, to see that it works well in practice. Difficulties for both the interviewer and the respondent should be recorded The questionnaire should then be revised in the light of the pilot’s findings. Sometimes a second pilot will be needed Once the questionnaire has been decided on, it needs to be tried out or piloted in a small number of households: perhaps 20 or so. Any difficulties should be recorded as they occur: it is just as important to record problems faced by the interviewer, as it is to record problems being faced by the respondent. These problems could be ones of phrasing of the question. problems of language if the questionnaire has been translated from English say, problems of using too much jargon or legal terms not readily understood. All the comments should be returned to the person designing the questionnaire who should then revise it in the light of the findings. If there are a lot of changes then the questionnaire should be piloted again.

Conduct of the survey The following practical aspects need to be fully planned for: Instructions to the interviewers should be drafted which are clear and unambiguous The conduct of the survey is either sub contracted to a competent private agency or a field force of interviewers recruited In either case the field force must be trained sufficiently to be able to carry out the survey competently. As it is important that a particular number of household is visited successfully, over sampling is necessary based upon the likely response rate: whereas 60-70% response rate is usually looked for, the local CSO may advise that response rate is likely to be lower. The conduct of the survey is a complex matter which needs to be carefully planned. The key is to have a good field force that is experienced in conducting surveys. Such a field force could be employed by the CSO, by a market research agency, or recruited specially for the purpose. Instructions for the interviewers need to be clear and unambiguous and cover all likely eventualities. If a sample of 1,000 is required as an example, then it will be necessary to over sample, as not all households will response positively. The usual response rate is around 60-70%, although this may require interviewers to be quire persistent with some householders, who may be out at work most of the time. However, the CSO or local market research agencies will be able to give their own estimates of the likely response rate. Thus, if the response rate is thought to be 60% and 1,000 positive responses are wanted, you need to sample nearly 1,700 households of which around 700 will not respond positively.

Analysis of the survey Experience has shown that questionnaires with open-ended questions are much less easy to analyse that closed questions that can be coded in advance. Although special survey analysis packages can be used, the most cost-effective way to produce the results is to: produce simple distributions for each variable create an EXCEL spreadsheet with households as rows and coded answers to questions as columns This will enable an analyst trained and experienced in EXCEL analysis to produce two- and three way tables swiftly using the standard EXCEL pivot table features. The design of the questionnaire should reflect the fact that the analysis should be as straightforward as possible. Experience has shown that questionnaires with open-ended questions are much less easy to analyse that closed questions that are pre-coded. Special survey analysis packages can be used if the agency conducting the survey is familiar with these. However, an alternative cost-effective way to produce the results is to: produce simple distributions for each variable create an EXCEL spreadsheet with households as rows and coded answers to questions as columns This will enable an analyst trained and experienced in EXCEL analysis to produce two- and three way tables swiftly using the standard EXCEL pivot table features.