Authors' objectives:

A selective, narrative literature review was conducted to identify current best practice with respect to the design and conduct of questionnaire surveys, including theories of respondent behaviour, 'expert opinion' and high-quality evidence from experimental studies. The principal foci were:

- modes of survey administration (various forms of interviewer administration and self-completion) - question wording, choice of response formats, and question sequencing - questionnaire formatting and other aspects of presentation - techniques for enhancing response rates, with particular emphasis on postal surveys.

Authors' results and conclusions: Mode of administration: The two principal modes of administration are self-completion and interviewer administration. Evidence from identified studies provided no consistent picture of the superiority of any one mode in terms of the quantity or quality of the response, or the resources required. Question wording and sequencing: Evidence from identified studies supported the notion that question wording and framing, including the choice and order of response categories, can have an important impact on the nature and quality of responses. The conventional wisdom with respect to question ordering is that general questions should precede specific questions; evidence from a number of primary studies supported this assertion. Questionnaire appearance: Through careful attention to the design and layout of questionnaires, the risk of errors in posing and interpreting questions and in recording and coding responses can be reduced, and potential inter-rater variability can be minimised. Evidence from experimental and quasi-experimental studies on aspects of questionnaire appearance was scanty. However, a number of articles were identified that outlined a theoretical basis to aspects of design, which suggested that questionnaire appearance can influence respondents' decisions at several stages, from arousal of interest in questionnaire completion, through task evaluation, to initiation and monitoring of the process of completion. There is a need for consistency in the presentation of visual information and an understanding and application of 'graphic non-verbal language' (i.e. the spatial arrangement of information and other visual phenomena such as colour and brightness). Enhancing response rates: High survey response rates are desirable because they increase the precision of parameter estimates and reduce the risk of non-response bias. Many factors may combine to influence the decision of a recipient of a questionnaire to respond. Potential respondents must have both the means to complete the questionnaire and the will to do so; the perceived costs of responding must not exceed the benefits. 'Saliency' - the apparent relevance, importance and interest of the survey to the respondent - is a very important influence on response rates. Fortunately, health-related surveys are likely to be perceived as salient. Perhaps surprisingly, questionnaire length appears to be less important. The number of contacts made with sampled individuals is another powerful factor. Some researchers advocate prenotification, so that recipients are primed for the arrival of the questionnaire. Almost all experts in survey design advocate the use of reminders, a recommendation supported by evidence from primary studies. Other factors that have been shown to influence response rates include making a self-interest/utility appeal to the respondent and the use of incentives (particularly enclosed monetary incentives). Perhaps surprisingly, anonymity has not been demonstrated to have any consistent effects on the rate or quality of response.

Authors' recommendations: The heterogeneity of findings indicates that there can be no universal recommendations on best practice in respect of questionnaire design and survey conduct. Rather, individual survey researchers need to take into account the aims of the particular study, the population under investigation and the resources available; trade-offs between the ideal and the possible are likely to be needed. However, some general principles can be offered. The principal objective should always be to collect reliable, valid and unbiased data from a representative sample, in a timely manner and within given resource constraints. In choosing a mode of questionnaire administration, consideration needs to be given to the availability of an appropriate sampling frame, anticipated response rates, the potential for bias from sources other than non-response, acceptability to the target population, the time available, the financial budget, and the availability of other resources (e.g. skills or equipment). In formulating questions and response categories, and in determining question order, researchers should bear in mind that survey respondents employ a wide range of cognitive processes in formulating their responses. To minimise bias and to reduce spurious inter-respondent variation, careful attention must be given to these issues. The "task analysis" model, the theory of social exchange and theories of perception and cognition should inform decisions regarding the physical design of questionnaires, as well as strategies for delivering and returning them. The aim should be to enhance the perceived and actual benefits of responding and to minimise the perceived and real costs. The effort required to interpret questions and provide responses should be made as easy as possible. Strategies for reducing the monetary cost to respondents include the use of prepaid return envelopes and the provision of financial incentives (unless ethical imperatives preclude the latter).

Authors' methods: Review, expert opinion

Project Status: Completed

URL for project: http://www.hta.ac.uk/937

Year Published: 2001

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: England, United Kingdom

Organisation Name: NIHR Health Technology Assessment programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk

Copyright: 2001 Queen's Printer and Controller of HMSO