Authors' objectives:

The aim of this study was to review the use of case series in National Institute for Clinical Excellence (NICE) Health Technology Assessment (HTA) reports, to review systematically the methodological literature for papers relating to the validity of aspects of case series design, and to investigate characteristics and findings of case series using examples from the UKs Health Technology Assessment programme.

Authors' results and conclusions: There was no consensus on which case series to include in HTAs, how to use them or how to assess their quality, despite them being used in 30% of NICE HTAs. No previous studies empirically investigating methodological characteristics of case series were found. However, it is possible that the search strategy failed to find relevant studies. Poor reporting of case series characteristics severely constrained analysis and there were insufficient data to investigate all the hypotheses. Findings were not consistent across the different topics and were subject to considerable uncertainty. All the examples in our analysis were surgical interventions, which are prone to additional confounding factors due to difficulties of standardisation compared with drug treatment. Our findings may not be generalisable outside the interventions studied. The case series reports included generally exhibited poor reporting of methodological characteristics. This constrained our analysis. The use of several methods of analysis has led to apparently discrepant results. Given the number of analysis performed, the usual level of significance (p = 0.05) should be viewed with caution. The most important limitation of this study is the small number of cases on which the findings are based. The results are therefore tentative and should be viewed with caution.

Authors' recommendations: Case series are incorporated in a significant proportion of health technology assessments. Quality criteria have been used to appraise the quality of case series and decide on their inclusion in reviews of studies using this design. In this small series of case studies drawn from HTAs carried out for the NHS HTA programme, little evidence was found to support the use of many of the factors included in quality assessment tools. Importantly, no relationship was found between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, such as blinding of outcome measurement, but these were not shown consistently across the small number of examples studied. This study is based on a very small sample of studies and should therefore be considered as exploratory. Further investigation of the relationship between methodological features and outcome is justified given the frequency of use of case series in health technology assessments. Further research into the methodological features of case series and their outcome is justified in a wider sample of technologies and larger sets of case series. Value of information analyses including case series could be explored. Further exploration of the differences between case series and randomised controlled trial results, preferably using registry or comprehensive case series data, would be valuable.

Authors' methods: Systematic review

Project Status: Completed

URL for project: http://www.hta.ac.uk/1371

Year Published: 2005

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: England, United Kingdom

Organisation Name: NIHR Health Technology Assessment programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk

Copyright: 2009 Queen's Printer and Controller of HMSO