Authors' recommendations: Based on our review of the literature, simulation studies, and our case study, we conclude the following concerning the use of Bayesian statistical approaches in CMS policy- and decisionmaking.1. CMS should consider claims about differential subgroup effects only if they are accompanied by a formal statistical test for interaction.a.Claims about differential subgroup effects based on stratified analysis should only be considered as exploratory. These analyses are compromised by the small sample sizes and post hoc decisions regarding the number of tested subgroups.b.Subgroup effects observed in a specific trial should be placed into context by using a statistical model that combines2information across trials and across subgroups. The random-effects/hierarchical models do both.2.To increase the statistical power to detect those interactions that in fact exist, consider using all sources of data in order to stipulate within the statistical model which types of interaction are likely. For example, observational data and expert opinion might suggest that if an interaction is present it will take the form of decreasing ICD efficacy with increasing burden of disease3.Base study design and decisionmaking only on those subgroup effects that are likely to be strong. The power to detect interactions is not universally high, and focusing attention on the most likely candidates will limit the number of subgroups that are analyzed, and thus limit the pernicious effects of random variation.4.If the trial-based data are sufficient, do not directly combine trial-based data with information from other sources such as observational data and expert opinion. In this case the objective data are sufficient, and there is no need to utilize subjective information. Instead, use these other sources as informal sources of validation, and also to help design the statistical model for the trials (see below).5.When little or no trial-based information about a subgroup is available, consider the use of other data (e.g., trial-based information from other subgroups, observational data, expert opinion) in order to specify a prior distribution. Unless special circumstances such as small patient pools are present, do not use this information to make final decisions about efficacy within the subgroups in question, but instead use this information to plan further studies. This suggests that the more controversial applications of Bayesian methodology should be reserved for those situations in which the decisionmaker has no other choice, and should, in any case, not be considered definitive.6.Claims based on Bayesian methods should provide sensitivity analysis to the assumed priors. While for large trials the results are not sensitive to prior choices, this is not the case for small size trials. It is therefore important to demonstrate through sensitivity analyses how the choice of the prior impacts (or does not impact) the findings.

Project Status: Completed

URL for project: http://www1.cms.gov/mcd/viewtechassess.asp?from2=viewtechassess.asp&where=index&tid=65&

Year Published: 2009

URL for published report: http://www.ahrq.gov/clinic/ta/comments/bayesian/

English language abstract: An English language summary is available

Publication Type: Not Assigned

Country: United States

Organisation Name: Agency for Healthcare Research and Quality

Contact Address: Center for Outcomes and Evidence Technology Assessment Program, 540 Gaither Road, Rockville, MD 20850, USA. Tel: +1 301 427 1610; Fax: +1 301 427 1639;

Contact Name: martin.erlichman@ahrq.hhs.gov

Contact Email: martin.erlichman@ahrq.hhs.gov

Copyright: Agency for Healthcare Research and Quality (AHRQ)