Authors' objectives:

To identify the factors that affect the decisions that emerge from consensus development methods.

To assess the implications of the findings for the development of clinical guidelines.

To recommend further methodological research for improving the use of consensus development methods as a basis for guideline production.

Authors' results and conclusions: Setting the task or question to be addressed Cues included in scenarios must be selected with care. As well as reviewing the relevant literature, clinicians in the consensus group should give their opinions (most usefully in the first round) about which cues are important. Doing so may help maintain their participation and help them justify their judgements. [C] Contextual cues included in scenarios are as important as ones specific to the topic at issue, and they should be made explicit. [B] It must be decided whether to focus on ways of managing a specific condition or on indications for using an intervention. If the focus is on an intervention, care should be taken about how to deal with other relevant interventions. [C] Is a global judgement elicited, or is an attempt made to break the judgement down into probability and utility estimates? Although there are theoretical advantages to the latter, it is likely to be a more difficult task for participants and it may not enhance judgements. [C] Inclusion of all possible scenarios may increase comprehensiveness, but if many of the scenarios never occur in practice, the increased burden on the respondents may not be justified by the limited value of the information provided. Judgements of scenarios which never or rarely occur in practice may be less reliable. [B] Requiring participants to judge what may be seen as numerous irrelevant scenarios may alienate them from the task. [C] Selecting the participants Within defined specialist or professional categories, the selection of the particular individuals is likely to have little impact on the decision of a group of sufficient size. To enhance the credibility and widespread acceptance of the guidelines, the participants should reflect the full range of key characteristics of the population that it is intended to influence. Selection should be seen to be unbiased. [C] To define common ground and maximise areas of agreement, groups should be homogeneous; to identify and explore areas of uncertainty, a heterogeneous group is appropriate. [B] In judgements of clinical appropriateness, the most influential background factor is the particular medical specialty. Specialists tend to favour the interventions with which they are most familiar. Consensus-based guidelines should therefore be interpreted in the context of the specialty composition of the group. [A] Choosing and preparing the scientific evidence A review of research-based information should be provided to all participants at an early stage. Participants should be encouraged to bring the review and any personal notes to the group sessions as memory aids. [B] Information presented in a synthesised form (e.g. tables) is more likely to be assimilated. Participants may be more likely to use information that is presented in an accessible format. Information tabulated so as to increase the salience of the dimensions to be used for making judgements is more likely to be processed in this manner. [C] Methodologists should be involved in conducting any literature review. [C] Grading the quality of studies using a reliable method may mitigate the biases of the reviewers somewhat, but may not eliminate them. [B] Structuring the interaction With NGTs and the Delphi method, two or more rating rounds are likely to result in some convergence of individual judgements, though it is unclear whether this increases the accuracy of the group decision. [A] With the Delphi method, it is advisable to feed back reasons or arguments as well as measures of central tendency or dispersion. [B] Efforts should be made to mitigate the effects of status of participants (which can affect their contribution to and influence within a group). [B] A comfortable environment for meetings is likely to be preferred by participants and to be conducive to discussion. [C] A good facilitator will enhance consensus development and can ensure that the procedure is conducted properly. [C] Methods of synthesising individual judgements An implicit approach to aggregating individual judgements may be adequate for establishing broad policy guidelines. More explicit methods based on quantitative analysis are needed to develop detailed, specific guidelines. [C] The more demanding the definition of agreement, the more anodyne the results will be. If the requirement is too demanding, either no statements will qualify or those that do will be of little interest. [C] Differential weighting of individual participants' views produces unreliable results unless there is a clear empirical basis for calculating the weights. [B] The exclusion of individuals with extreme views (outliers) can have a marked effect on the content of guidelines. [A] There is no agreement as to the best method of mathematical aggregation. [B] Reports of consensus development exercises should include an indication of the distribution or dispersal of participants' judgements, not just the measure of central tendency. In general, the median and the inter- quartile range are more robust than the mean and standard deviation. [A]

Authors' recommendations: Priorities for future research What impact does the framing or presentation of the question have on individual judgements? In what form and how inclusive should scenarios be? How does the extent of heterogeneity of a group affect the process and outcome? What effect does research-based information have on individual and on group judgements? Does the effect depend on the amount of information or how it is presented? What effect does the method of feedback of participants' views have on group judgement?

Authors' methods: Review

Project Status: Completed

URL for project: http://www.hta.ac.uk/944

Year Published: 1998

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: England, United Kingdom

Organisation Name: NIHR Health Technology Assessment programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk

Copyright: 2009 Queen's Printer and Controller of HMSO