Authors' objectives:

Methodological research has few well-defined tools and processes analogous to those available for reviews and data collection in substantive health technology assessment.

This project was set up to obtain researchers and others views on the innovative projects on research methodology under the NHS Health Technology Assessment Programme and the usefulness of the research.

The study was intended to span both epistemological and management issues.

The following issues were explored: - the degree to which researchers would feel constrained by the Cochrane approach to systematic reviews when undertaking reviews of a methodological nature - whether methodological projects may require exceptional design and management arrangements, in view of their novelty, subjectivity and complexity - whether researchers would seek out other methods, in addition to undertaking reviews of argument, as a means of extending their understanding of methodological issues (there may be three categories of research methods in methodology: reviews of methodological argument, studies that use the literature as a source of data, and research that collects new primary data) - whether the Methodology Programme overall can be considered a success.

Authors' recommendations: Reflections on the findings: These studies were commissioned at a time of high enthusiasm for systematic reviews and meta-analysis, resulting in both epistemological and practical problems for some of those undertaking them. The importance of systematicity rather than exhaustiveness needs to be recognised. The question of whether these reviews should be seen primarily as data collection or thinking exercises pervaded the interviews and has important practical implications. Contrary to our expectations, little was made (in this sample) of additional research methods, with few attempts at triangulation. Researchers were not opposed to changing methods in principle but did not do so in practice. Researchers seemed very conscious of the problem of bias and undertook an impressive range of steps to reduce its impact. The organisation and management demands on these projects were not notably different from those for other studies, including the need to plan carefully from the outset and for close research management by senior staff, but some issues may be heightened in this context. The Programme was largely seen as a success, covering a wide range of issues and helping to develop a pool of researchers familiar with the field. The website analysis provided a quick illustration of the considerable interest shown in methodological and other reports. The significance of the Methodology Programme should be seen to lie not simply in the reports produced, but in the diffusion of knowledge it facilitated. Recommendations for researchers: The conduct of research: Researchers should consider methods beyond the review of ideas and even he review of data, for instance, networking and other means of primary data collection (e.g. methodological studies attached to primary health technology assessment). Because systematic reviews in this context are very different from traditional Cochrane type' reviews, methodological researchers should not try to chase every reference, but ensure that they search widely (i.e. consider disparate databases and sources). Some overlap of the various stages of research ; searching, analysis, synthesis and writing ; should be encouraged because this can help to clarify the nature of the research. Researchers should publicise their studies early on, to help to short-circuit extensive search processes and stimulate ideas. All studies should include a short summary of key findings, which should include practical solutions to identified problems, to assist future researchers. Reducing bias: A variety of safeguards to reduce potential bias should be built in to research, including the establishment of a steering group, multidisciplinary teams, peer reviewing of applications and final reports, a report for the commissioning body, and a clear intention to publish in widely disseminated journals. The efficient management of research: Senior staff need to be closely involved throughout the research, both to assist with data analysis and to maintain good research management; this should be reflected in the costing of grants. A clear plan of action and research timetable should be developed, including a plan for the report writing, with a preliminary structure to influence both data collection and analysis. Multisite projects should be considered carefully before they are set up, with respect to both the logistics and the willingness of the parties to work together.

Authors' methods: Survey

Project Status: Completed

URL for project: http://www.hta.ac.uk/1102

Year Published: 2001

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: England, United Kingdom

Organisation Name: NIHR Health Technology Assessment programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk

Copyright: 2009 Queen's Printer and Controller of HMSO