1. Quality and purpose: quality in quantitative methods
Angela Dale
University of Manchester

2. Quality in survey methods
A survey is of adequate quality if it can answer specified question with an adequate degree of accuracy
it is fit for purpose
If it can provide an answer at a 1% level of confidence when only 5% is required, then it may not be cost effective

3. Standards for surveys
Random sample – best bet of avoiding bias in sample; but need adequate sampling fame
Adequate numbers for required subgroups – depends on level of accuracy needed
Good response rate – but extent of bias is crucial
Questionnaire must be well developed
well designed questions; good flow; salient

4. Can a scoring system work?
A scoring system can indicate overall strength on all these dimensions, but:
Higher quality costs more
Increased spend may have diminishing returns
But there will be a level below which a survey would have very low value/low credibility
Need to find a way of assessing quality against requirement

5. Analysis issues
Results based on analysis need to reflect the quality of the survey
Importance of making clear the level of accuracy of the results
The quality of the analysis is also of great importance
An excellent survey may be badly analysed ,
methods may be used poorly; interpretations incorrect
inappropriate assumptions about causality may be made
Claims may go beyond what the data can support

6. Can a poor survey have value?
A very poor survey may still have value:
if analysed with care and weaknesses recognised
If only very limited conclusions, that can be justified, are drawn
Where there is no better alternative

7. Conclusions
It is of value to have a reference framework for assessing quality
But equally important to use it critically and with care
The key to quality lies not just in good data but in ensuring that claims made can be supported by the data