Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses

Stefan Schandelmaier, Matthias Briel, Ravi Varadhan, Christopher H. Schmid, Niveditha Devasenapathy, Rodney A. Hayward, Joel Gagnier, Michael Borenstein, Geert J.M.G. van der Heijden, Issa J. Dahabreh, Xin Sun, Willi Sauerbrei, Michael Walsh, John P.A. Ioannidis, Lehana Thabane, Gordon H. Guyatt

Research output: Contribution to journalArticlepeer-review

Abstract

BACKGROUND: Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN). METHODS: To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification. RESULTS: The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument. INTERPRETATION: The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.

Original languageEnglish (US)
Pages (from-to)E901-E906
JournalCMAJ
Volume192
Issue number32
DOIs
StatePublished - Aug 10 2020

ASJC Scopus subject areas

  • Medicine(all)

Fingerprint Dive into the research topics of 'Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses'. Together they form a unique fingerprint.

Cite this