Authors' objectives:

This paper has been prepared to provide a basis for discussion on the components of health technology assessment programs and approaches to measuring their effectiveness. There is increasing interest in better defining the role played by HTA agencies and in obtaining advice on how they might most appropriately be structured and operated. It is with the spirit of moving toward a continual improvement of activities that this paper has been prepared.

An overall theme of the paper is that there are a number of determinants of the effectiveness of an HTA program, only some of which are within the control of those who operate the HTA function. Some specific reference is made to the Health Technology Assessment Unit (HTAU) of the Alberta Heritage Foundation for Medical Research, but a number of the issues discussed should be more widely applicable to other HTA organizations.

The paper is not intended as a comprehensive guide to measuring effectiveness of HTA programs. Rather, it seeks to raise issues and provide managers with suggestions that may be helpful in developing specific administrative guidance for individual programs. It can be thought of as a self assessment tool to help identify opportunities for future improvement.

In the following discussion the term HTA program is used both to describe the staff and structure in organisations that conduct HTA and the wider picture that includes various other external determinants of effectiveness.

Authors' recommendations: The health technology assessment process is complex. This paper has identified some key features of HTA programs and suggested directions for assessment of their effectiveness. There is no easy answer to the question of how to assess effectiveness of HTA programs. Individual programs will vary considerably in their structure and in their values and the level of importance that are placed on different attributes. There is also a question of the perspective from which effectiveness is measured. The discussion in this paper has tended to reflect the perspective of a manager from within an HTA program. Different emphases and perceptions on various issues may emerge from other stakeholders. In practice, HTA involves a series of trade offs. HTA programs will typically be dealing with many topics, which will have differing priorities and require varying levels of assessment. The potential ideal of a comprehensive, methodologically rigorous assessment has to be balanced by the realities of available resources, data and time, and put into further context by the question asked of the HTA program and the likely impact of the subsequent HTA product. It is not possible to specify a definitive approach to management and measurement of complex processes such as these, or of relationships with other organizations that will to a large extent determine the impact of HTA products. The discussion given earlier suggests some areas that might be considered when appraising the effectiveness of HTA programs. Measures of effectiveness ought to include at least some consideration of activity, continuity, relevance, quality and impact. It will be a matter for HTA program managers to determine how these might be brought together for routine administrative purposes and how the information obtained should be used. The level of detail needed will depend on what these measures of effectiveness are to be used for and by whom internal management, accountability to governance, or advice to the external environment. Keeping assessment of HTA programs focused, concise, practical and useful will continue to present challenges.

Project Status: Completed

Year Published: 2003

URL for published report: https://www.ihe.ca/advanced-search?type=1020

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: Canada

Organisation Name: Institute of Health Economics

Contact Address: 1200, 10405 Jasper Avenue, Edmonton, Alberta, Canada, T5J 3N4. Tel: +1 780 448 4881; Fax: +1 780 448 0018;

Contact Name: djuzwishin@ihe.ca

Contact Email: djuzwishin@ihe.ca

Copyright: <p>Alberta Heritage Foundation for Medical Research (AHFMR)</p>