Use of Bayesian techniques in randomized clinical trials: a CMS case study

Sanders GD, Inoue L, Samsa G, Kulasingam S, Matchar D
Record ID 32007000574
Authors' recommendations: Based on our review of the literature, simulation studies, and our case study, we conclude the following concerning the use of Bayesian statistical approaches in CMS policy- and decisionmaking.1. CMS should consider claims about differential subgroup effects only if they are accompanied by a formal statistical test for interaction.a.Claims about differential subgroup effects based on stratified analysis should only be considered as exploratory. These analyses are compromised by the small sample sizes and post hoc decisions regarding the number of tested subgroups.b.Subgroup effects observed in a specific trial should be placed into context by using a statistical model that combines2information across trials and across subgroups. The random-effects/hierarchical models do both.2.To increase the statistical power to detect those interactions that in fact exist, consider using all sources of data in order to stipulate within the statistical model which types of interaction are likely. For example, observational data and expert opinion might suggest that if an interaction is present it will take the form of decreasing ICD efficacy with increasing burden of disease3.Base study design and decisionmaking only on those subgroup effects that are likely to be strong. The power to detect interactions is not universally high, and focusing attention on the most likely candidates will limit the number of subgroups that are analyzed, and thus limit the pernicious effects of random variation.4.If the trial-based data are sufficient, do not directly combine trial-based data with information from other sources such as observational data and expert opinion. In this case the objective data are sufficient, and there is no need to utilize subjective information. Instead, use these other sources as informal sources of validation, and also to help design the statistical model for the trials (see below).5.When little or no trial-based information about a subgroup is available, consider the use of other data (e.g., trial-based information from other subgroups, observational data, expert opinion) in order to specify a prior distribution. Unless special circumstances such as small patient pools are present, do not use this information to make final decisions about efficacy within the subgroups in question, but instead use this information to plan further studies. This suggests that the more controversial applications of Bayesian methodology should be reserved for those situations in which the decisionmaker has no other choice, and should, in any case, not be considered definitive.6.Claims based on Bayesian methods should provide sensitivity analysis to the assumed priors. While for large trials the results are not sensitive to prior choices, this is not the case for small size trials. It is therefore important to demonstrate through sensitivity analyses how the choice of the prior impacts (or does not impact) the findings.
Project Status: Completed
Year Published: 2009
English language abstract: An English language summary is available
Publication Type: Not Assigned
Country: United States
MeSH Terms
  • Bayes Theorem
  • Research Design
Organisation Name: Agency for Healthcare Research and Quality
Contact Address: Center for Outcomes and Evidence Technology Assessment Program, 540 Gaither Road, Rockville, MD 20850, USA. Tel: +1 301 427 1610; Fax: +1 301 427 1639;
Contact Name:
Contact Email:
Copyright: Agency for Healthcare Research and Quality (AHRQ)
This is a bibliographic record of a published health technology assessment from a member of INAHTA or other HTA producer. No evaluation of the quality of this assessment has been made for the HTA database.