Victimization surveys: 3 common mistakes to avoid

January 1st, 2016

by Heather Sutton

Developing evidence-based policies, particularly in the developing world, can be difficult when reliable data are not available. Many crimes go unreported by the public, or unrecorded by the police. Victimization surveys are increasingly recognized as useful tools, but these surveys can be complex and costly, which can lead to cutting corners and producing low quality data. Because victimization surveys have unique requirements, it is easy for non-experts to wind up with a survey that may look like a crime survey, but cannot answer many of the important questions it was developed for.

Guidelines for anyone planning to fund or implement these surveys can be found in Victimization Surveys 101: Recommendations for Funding and Implementing a Victimization Survey. Here are some common mistakes to avoid:

• Mistaking an opinion survey for a victimization survey

A victimization survey is not the same as a “citizen security” survey. The latter is often a survey of perceptions and attitudes about crime, instead of actual individual experiences of crime. It is common to ask about both opinions and experiences in victimization surveys, but there are important differences between a victimization survey with some opinion questions and the opposite. Everyone has an opinion, so opinion surveys are much cheaper than victimization surveys. You don’t need as big a sample size to generate reliable estimates of public opinion on crime (explained further under number 3). If you’re spending the money to do a victimization survey, you should do a victimization survey. Otherwise, you shouldn’t call your survey a victimization survey and should not attempt to estimate prevalence of different crimes.

• Deviating from recommended techniques and wording

Many specific survey techniques have been developed by experts in victimization surveys and it makes sense to follow them (learn more). Some techniques – such as those to increase disclosure, the placement of screener questions, question sequencing and particular question wording – have been perfected overtime. Don’t reinvent the wheel. Adopt many of these techniques and take questions directly from vetted instruments like the International Crime Victims Survey. Have a good reason if deviating from other surveys in terms of question wording or sequencing, and understand the implications (losing comparability).

• Producing low quality surveys with inadequate sample sizes

Even where crime is high, it is still a fairly rare. This means that the victimization survey sample size (the amount of respondents surveyed) needs to be much larger than an average survey. The rarer the crime, the harder it is to measure. Violent crimes are less common than property crimes. So it takes a bigger sample to estimate assault than burglary. It takes an even bigger sample to ask for more information about a given crime (i.e. weapons used, relation between the victim and offender, etc.). Many so called victimization surveys include unusable questions on violent crimes based on their sample size. You can include many detailed questions, but you probably won’t have a big enough sample to use them all. Getting the right sample size means victimization surveys are far more expensive than other surveys (typically several million $US). A low budget negatively affects the outcome; low quality of data collected which is less compelling as ‘evidence’. Unfortunately, many people and government officials don’t know how to tell the difference between a good study and a poor one. Funders can really make a unique contribution by opting for good surveys of a smaller area (such as a capital city) over weak surveys of an entire country.

Tags: attitudes, domestic violence and Caribbean., gender, violence, women