Fixed vs random effects meta-analysis in rare event studies: The Rosiglitazone link with myocardial infarction and cardiac death

Meta-analyses can be powerful tools to combine the results of randomized clinical trials and observational studies to make consensus inferences about a medical issue. It will be demonstrated that a common practice of testing for homogeneity of effect size, and acting upon the inference to decide between fixed vs random effects, can lead to potentially misleading results. A by-product of this paper is a new ratio estimator approach to random effects meta-analysis of a large set of studies with low event rates. As a case study, we shall use the recent Rosiglitazone example, where diagnostic testing failed to reject homogeneity, leading the investigators to use fixed effects. The results for the fixed and random effects analyses are discordant. In the fixed (random) effects analysis, the p-values for myocardial infarction were 0.03 (0.11) while those for cardiac death were 0.06 (0.0017). Had the fixed effects analysis controlled the study error for multiple testing via a Bonferonni correction, the joint 95+ per cent confidence rectangle for the two outcomes would have included odds ratios of (1.0, 1.0). For the Rosiglitazone example, random effects analysis, where all studies receive the same weight, is the superior choice over fixed effects, where two large studies dominate.