TY - JOUR
T1 - Development of the instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses
AU - Schandelmaier, Stefan
AU - Briel, Matthias
AU - Varadhan, Ravi
AU - Schmid, Christopher H.
AU - Devasenapathy, Niveditha
AU - Hayward, Rodney A.
AU - Gagnier, Joel
AU - Borenstein, Michael
AU - van der Heijden, Geert J.M.G.
AU - Dahabreh, Issa J.
AU - Sun, Xin
AU - Sauerbrei, Willi
AU - Walsh, Michael
AU - Ioannidis, John P.A.
AU - Thabane, Lehana
AU - Guyatt, Gordon H.
N1 - Publisher Copyright:
© 2020 Joule Inc. or its licensors
PY - 2020/8/10
Y1 - 2020/8/10
N2 - BACKGROUND: Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN). METHODS: To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification. RESULTS: The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument. INTERPRETATION: The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.
AB - BACKGROUND: Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN). METHODS: To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification. RESULTS: The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument. INTERPRETATION: The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.
UR - http://www.scopus.com/inward/record.url?scp=85089327341&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85089327341&partnerID=8YFLogxK
U2 - 10.1503/cmaj.200077
DO - 10.1503/cmaj.200077
M3 - Article
C2 - 32778601
AN - SCOPUS:85089327341
SN - 0820-3946
VL - 192
SP - E901-E906
JO - CMAJ
JF - CMAJ
IS - 32
ER -