Authors' objectives:

The specific objectives of the pilot study were to:

- demonstrate the benefits of using appropriate decision-analytic methods and value of information analysis - establish the feasibility and resource implications of applying these methods in a timely way, to inform the prioritisation process of the HTA programme - establish the resource implications of adopting these methods more widely within the NHS HTA programme - identify the most appropriate way to extend the use of these methods within the programmes prioritisation process.

Authors' results and conclusions: Although none of the research topics identified by NCCHTA met all of the original selection criteria for inclusion as case studies in the pilot, it was possible to construct appropriate decision-analytic models and conduct probabilistic analysis for each topic. In each case, the three core tasks were completed within the time-frame required by the existing HTA research prioritisation process. The brief case-study reports provided a description of the decision problem, a summary of the current evidence base and a characterisation of decision uncertainty in the form of cost-effectiveness acceptability curves. Estimates of value of information for the decision problem were presented for relevant patient groups and clinical settings, as well as the value of information associated with particular model inputs. The implications for the value of research in each of the areas were presented in general terms. Details were also provided on what the analysis suggested regarding the design of any future research in terms of features such as the relevant patient groups and comparators, and whether experimental design was likely to be required.

Authors' recommendations: - The pilot study showed that, even with very short timelines, it is possible to undertake DA-VOI that can feed into the priority-setting process that has developed for the HTA programme. - The use of DA-VOI requires relevant stakeholders to be clear, from an early point in the process, about the nature of the decision problem for which additional research is being considered. - DA-VOI also needs explicitness about which existing data should be used for the first part of the analysis, and how data that exhibit particular weaknesses should be down-weighted in the analysis. - There would be advantages to making the development of the vignette (a summary of the clinical problem and existing evidence) and the use of DA-VOI an integrated process. - It is estimated that each of the pilot studies undertaken required approximately 6 weeks whole-time equivalent researcher input, and this was made up of a mix of experience levels. This research activity needs to be spread out over a period of 1012 weeks, in part to allow for evidence acquisition. - One approach to the more extensive use of DA-VOI might involve working up a proportion of topics for DA-VOI once they have been identified for a vignette based on existing methods. These analyses would be presented to the panels, along with the vignettes, and they would provide feedback. At the PSG, there would be an analysis that directly addresses the question in the vignette and would include additional analysis to explore any concerns or issues raised by the panel. - Practical considerations about how to implement such methods into a priority-setting system, which has evolved in a particular way, are complex. These include appropriate levels of training for individuals on the relevant panels to achieve the most from DA-VOI, and how analyses of acceptable quality can be assembled in a timely way given limitations of time and skilled resources. - There needs to be some reflection on how the DA-VOI methods handle the heterogeneity and differing levels of quality in the evidence base. Greater use of sensitivity analysis may be a way of handling this problem. Consideration needs to be given to identifying useful scenarios and priorities for sensitivity analysis. This may be an iterative process based on concerns expressed by the panels. - There is a need to identify, and secure access to, relevant clinical experts early in the analysis period when the decision problem is being defined, the structure of the model is being established and relevant data are being identified. - If some degree of implementation of DA-VOI takes place within the HTA programme, careful evaluation and ongoing development will be essential.

Project Status: Completed

URL for project: http://www.hta.ac.uk/1335

Year Published: 2004

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: England, United Kingdom

Organisation Name: NIHR Health Technology Assessment programme

Contact Address: NIHR Journals Library, National Institute for Health and Care Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK

Contact Name: journals.library@nihr.ac.uk

Contact Email: journals.library@nihr.ac.uk

Copyright: 2009 Queen's Printer and Controller of HMSO