Authors' objectives:

Systematic review and meta-analytical methods are already common approaches to the assessment of health technology and related areas, and increasing adoption of such approaches may be foreseen, in part in response to increasingly wide emphasis on evidence-based approaches to medicine and health care. This report is intended:

1. To identify applications of systematic review and meta-analytical methods in Health Technology Assessment (HTA).

2. To promote further, appropriate use of such approaches in these areas of application.

3. To begin to identify priorities for further methodological developments in this field.

Authors' recommendations: Recommendations for good practice for the most part follow standard and widely agreed approaches. Greater latitude in the nature of studies potentially eligible for review, including non-randomised studies and the results of audit exercises, for example, may, however, be appropriate. The key stages are (with extensions and/or less widely agreed aspects in parentheses): 1. Specification in a protocol of the objectives, hypotheses (in both biological and health care terms), scope, and methods of the systematic review, before the study is undertaken. 2. Compilation of as comprehensive a set of reports as possible of relevant primary studies, having searched for all potentially relevant data, clearly documenting all search methods and sources. 3. Assessment of the methodological quality of the set of studies (the method being based on the extent to which susceptibility to bias is minimised, and the specific system used reported). Any selection of studies on quality or other criteria should be based on clearly stated a priori specifications. The reproducibility of the procedures in 2 and 3 should also be assessed. 4. Identification of a common set of definitions of outcome, explanatory and confounding variables, which are, as far as possible, compatible with those in each of the primary studies. 5. Extraction of estimates of outcome measures and of study and subject characteristics in a standardised way from primary study documentation, with due checks on extractor bias. Procedures should be explicit, unbiased and reproducible. 6. Perform, where warranted by the scope and characteristics of the data compiled, quantitative synthesis of primary study results (meta-analysis) using appropriate methods and models (clearly stated), in order to explore and allow for all important sources of variation (e.g. differences in study quality, participants, in the dose, duration, or nature of the intervention, or in the definitions and measurement of outcomes). This will often involve the use of mixed/hierarchical models, including fixed covariates to explain some elements of between-study variation, in combination with random effects terms. 7. Performance of a narrative or qualitative summary, where data are too sparse, or of too low quality, or too heterogeneous to proceed with a statistical aggregation (meta-analysis). In such cases the process of conduct and reporting should still be rigorous and explicit. 8. Exploration of the robustness of the results of the systematic review to the choices and assumptions made in all of the above stages. In particular, the following should be explained or explored: the impact of study quality/inclusion criteria the likelihood and possible impact of publication bias the implications of the effect of different model selection strategies, and exploration of a reasonable range of values for missing data from studies with uncertain results. 9. Clear presentation of key aspects of all of the above stages in the study report, in order to enable critical appraisal and replication of the systematic review. These should include a table of key elements of each primary study. Graphical displays can also assist interpretation, and should be included where appropriate. Confidence intervals around pooled point estimates should be reported. 10. Appraisal of methodological limitations of both the primary studies and the systematic review. Any clinical or policy recommendations should be practical and explicit, and make clear the research evidence on which they are based. Proposal of a future research agenda should include clinical and methodological requirements as appropriate.

Authors' methods: Systematic review

Project Status: Completed

URL for project: http://www.hta.ac.uk/942

Year Published: 1998

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: England, United Kingdom

Organisation Name: NIHR Health Technology Assessment programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk

Copyright: 2009 Queen's Printer and Controller of HMSO