Authors' objectives:

To contribute to making searching for Technology Assessment Reports (TARs) more cost-effective by suggesting an optimum literature retrieval strategy, based on empirical data obtained from a sample of recent TARs, which balances comprehensiveness and efficiency.

Authors' results and conclusions: Sources searched: The median number of sources searched per TAR was 20, and the range was from 13 to 33 sources. Six sources (CCTR, DARE, EMBASE, MEDLINE, NHS EED and sponsor/industry submissions to NICE) were used in all reviews. Clinical effectiveness studies: There were 424 studies in total. The publication types were: published 80%, meeting abstracts 11.3% and unpublished 8.7%. Eighty per cent of reviews included at least one abstract or unpublished study (60% included at least one abstract and 50% included at least one unpublished study). The median number of studies included per TAR was 19.5 (range 241). The median number of participants included per TAR was 2787 (range 6997,570). Evidence from non-randomised controlled trial (RCT) studies was used in 45% of TARs. The proportion of studies classified either as published in full or as abstracts, and found indexed in the following databases, was: MEDLINE 82.7%, EMBASE 78.6% and CCTR 50.1%. The cumulative percentage of studies found after searching these three databases was 87.3%. Adding SCI, BIOSIS and ASCO Online increased this to 98.2%. Eighty-seven per cent of studies were indexed in both MEDLINE and EMBASE. Cost-effectiveness studies: The 130 studies were classified as: published 73.1%, unpublished 23.8%, abstracts 1.5% and grey literature 1.5%. The median number of studies used was 4.0. The percentage of studies classified as either published in full or as abstracts, and found indexed in the following databases, was: MEDLINE 86.6%, EMBASE 86.6% and NHS EED 40.2%. The cumulative percentage of these studies found indexed after searching the three databases was 94.8%. Adding SCI and ASCO Online increased this to 97.9%. Studies used in the economic modelling: The 121 articles were classified as: published 50.4%, abstracts 5.0%, reference sources 17.4%, unpublished 17.4% and grey literature 9.8%. The median number of studies used for the 14 TARs that included an economic model was 9.0 per TAR. Search terms for identifying non-RCTs: A sensitive search filter, constructed for MEDLINE and using the search terms from the bibliographic records in the included studies, retrieved only 85% of the known sample. Therefore, it is recommended that when searching for non-RCT studies a search is done for the intervention alone, and records are then scanned manually for those that look relevant.

Authors' recommendations: Searching additional databases beyond the Cochrane Library (which includes CCTR, NHS EED and the HTA database), MEDLINE, EMBASE and SCI, plus BIOSIS limited to meeting abstracts only, is seldom effective in retrieving additional studies for inclusion in the clinical and cost-effectiveness sections of TARs (apart from reviews of cancer therapies, where a search of the ASCO database is recommended). A more selective approach to database searching would suffice in most cases and would save resources, thereby making the TAR process more efficient. However, searching non-database sources (including submissions from manufacturers, recent meeting abstracts, contact with experts and checking reference lists) does appear to be a productive way of identifying further studies.

Authors' methods: Primary research

Project Status: Completed

URL for project: http://www.hta.ac.uk/1346

Year Published: 2003

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: England, United Kingdom

Organisation Name: NIHR Health Technology Assessment programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk

Copyright: 2009 Queen's Printer and Controller of HMSO