A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury

W. P. Coleman, E. Benzel, D. W. Cahill, T. Ducker, F. Geisler, B. Green, M. R. Gropper, J. Goffin, P. W. Madsen, D. J. Maiman, S. L. Ondra, M. Rosner, R. C. Sasso, G. R. Trost, S. Zeidman

Research output: Contribution to journalArticle

Abstract

From the beginning, the reporting of the results of National Acue Spinal Cord Injury. Studies (NASCIS) II and III has been incomplete, leaving clinicians in the spinal cord injury (SCI) community to use or avoid using methylprednisolone in acute SCI on the basis of faith rather than a publicly developed scientific consensus. NASCIS II was initially reported by National Institutes of Health announcements, National Institutes of Health facsimifiles to emergency room physicians, and the news media. The subsequent report in the New England Journal of Medicine implied that there was a positive result in the primary efficacy analysis for the entire 487 patient sample. However, this analysis was in fact negative, and the positive result was found only in a secondary analysis of the subgroup of patients who received treatment within 8 hours, In addition, that subgroup apparently had only 62 patients taking methylprednisolone and 67 receiving placebo. The NASCIS II and III reports embody specific choices of statistical methods that have strongly shaped the reporting of results but have not been adequately challenged or or even explained. These studies show statistical artifacts that call their results into question. In NASCIS II, the placebo group treated before 8 hours did poorly, not only when compared with the methylprednisolone group treated before 8 hours but even when compared with the placebo group treated after 8 hours. Thus, the positive result may have been caused by a weakness in the control group rather than any strength of methylprednisolone. In NASCIS III, a randomization imbalance occurred that allocated a disproportionate number of patients with no motor deficit (and therefore no chance for recovery) to the lower dose control group. When this imbalance is controlled for, much of the superiority of the higher dose group seems to disappear. The NASCIS group's decision to admit persons with minor SCIs with minimal or no motor deficit not only enables statistical artifacts it complicates the interpretation of results from the population actually sampled. Perhaps one half of the NASCIS III sample may have had at most a minor deficit. Thus, we do not know whether the results of these studies reflect the severely injured population to which they have been applied. The numbers, tables, and figures in the published reports are scant and are inconsistently defined, making it impossible even for professional statisticians to duplicate the analyses, to guess the effect of changes in assumptions, or to supply the missing parts of the picture. Nonetheless, even 9 years after NASCIS II, the primary, data have not been made public. The reporting of the NASCIS studies has fallen far short of the guidelines of the ICH/FDA and of the Evidence-based Medicine Group. Despite the lucrative "off label" markets for methylprednisolone in SCI, no Food and Drug Association indication has been obtained. There has been no public process of validation. These shortcomings have denied physicians the chance to use confidently a drug that many were enthusiastic about and has left them in an intolerably ambiguous position in their therapeutic choices, in their legal exposure, and in their ability to perform further research to help their patients.

Original languageEnglish (US)
Pages (from-to)185-199
Number of pages15
JournalJournal of Spinal Disorders
Volume13
Issue number3
DOIs
StatePublished - 2000
Externally publishedYes

Fingerprint

Methylprednisolone
Spinal Cord Injuries
Placebos
National Institutes of Health (U.S.)
Artifacts
Physicians
Control Groups
New England
Evidence-Based Medicine
Random Allocation
Pharmaceutical Preparations
Population
Hospital Emergency Service
Medicine
Guidelines
Food
Therapeutics
Research

Keywords

  • Drug approval
  • Methylprednisolone
  • Spinal cord injufies
  • U.S. Food and Drug Administration

ASJC Scopus subject areas

  • Clinical Neurology
  • Surgery

Cite this

A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury. / Coleman, W. P.; Benzel, E.; Cahill, D. W.; Ducker, T.; Geisler, F.; Green, B.; Gropper, M. R.; Goffin, J.; Madsen, P. W.; Maiman, D. J.; Ondra, S. L.; Rosner, M.; Sasso, R. C.; Trost, G. R.; Zeidman, S.

In: Journal of Spinal Disorders, Vol. 13, No. 3, 2000, p. 185-199.

Research output: Contribution to journalArticle

Coleman, WP, Benzel, E, Cahill, DW, Ducker, T, Geisler, F, Green, B, Gropper, MR, Goffin, J, Madsen, PW, Maiman, DJ, Ondra, SL, Rosner, M, Sasso, RC, Trost, GR & Zeidman, S 2000, 'A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury', Journal of Spinal Disorders, vol. 13, no. 3, pp. 185-199. https://doi.org/10.1097/00002517-200006000-00001
Coleman, W. P. ; Benzel, E. ; Cahill, D. W. ; Ducker, T. ; Geisler, F. ; Green, B. ; Gropper, M. R. ; Goffin, J. ; Madsen, P. W. ; Maiman, D. J. ; Ondra, S. L. ; Rosner, M. ; Sasso, R. C. ; Trost, G. R. ; Zeidman, S. / A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury. In: Journal of Spinal Disorders. 2000 ; Vol. 13, No. 3. pp. 185-199.
@article{95437208807048d0a1fd1ef4ace6234c,
title = "A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury",
abstract = "From the beginning, the reporting of the results of National Acue Spinal Cord Injury. Studies (NASCIS) II and III has been incomplete, leaving clinicians in the spinal cord injury (SCI) community to use or avoid using methylprednisolone in acute SCI on the basis of faith rather than a publicly developed scientific consensus. NASCIS II was initially reported by National Institutes of Health announcements, National Institutes of Health facsimifiles to emergency room physicians, and the news media. The subsequent report in the New England Journal of Medicine implied that there was a positive result in the primary efficacy analysis for the entire 487 patient sample. However, this analysis was in fact negative, and the positive result was found only in a secondary analysis of the subgroup of patients who received treatment within 8 hours, In addition, that subgroup apparently had only 62 patients taking methylprednisolone and 67 receiving placebo. The NASCIS II and III reports embody specific choices of statistical methods that have strongly shaped the reporting of results but have not been adequately challenged or or even explained. These studies show statistical artifacts that call their results into question. In NASCIS II, the placebo group treated before 8 hours did poorly, not only when compared with the methylprednisolone group treated before 8 hours but even when compared with the placebo group treated after 8 hours. Thus, the positive result may have been caused by a weakness in the control group rather than any strength of methylprednisolone. In NASCIS III, a randomization imbalance occurred that allocated a disproportionate number of patients with no motor deficit (and therefore no chance for recovery) to the lower dose control group. When this imbalance is controlled for, much of the superiority of the higher dose group seems to disappear. The NASCIS group's decision to admit persons with minor SCIs with minimal or no motor deficit not only enables statistical artifacts it complicates the interpretation of results from the population actually sampled. Perhaps one half of the NASCIS III sample may have had at most a minor deficit. Thus, we do not know whether the results of these studies reflect the severely injured population to which they have been applied. The numbers, tables, and figures in the published reports are scant and are inconsistently defined, making it impossible even for professional statisticians to duplicate the analyses, to guess the effect of changes in assumptions, or to supply the missing parts of the picture. Nonetheless, even 9 years after NASCIS II, the primary, data have not been made public. The reporting of the NASCIS studies has fallen far short of the guidelines of the ICH/FDA and of the Evidence-based Medicine Group. Despite the lucrative {"}off label{"} markets for methylprednisolone in SCI, no Food and Drug Association indication has been obtained. There has been no public process of validation. These shortcomings have denied physicians the chance to use confidently a drug that many were enthusiastic about and has left them in an intolerably ambiguous position in their therapeutic choices, in their legal exposure, and in their ability to perform further research to help their patients.",
keywords = "Drug approval, Methylprednisolone, Spinal cord injufies, U.S. Food and Drug Administration",
author = "Coleman, {W. P.} and E. Benzel and Cahill, {D. W.} and T. Ducker and F. Geisler and B. Green and Gropper, {M. R.} and J. Goffin and Madsen, {P. W.} and Maiman, {D. J.} and Ondra, {S. L.} and M. Rosner and Sasso, {R. C.} and Trost, {G. R.} and S. Zeidman",
year = "2000",
doi = "10.1097/00002517-200006000-00001",
language = "English (US)",
volume = "13",
pages = "185--199",
journal = "Journal of Spinal Disorders",
issn = "1536-0652",
publisher = "Lippincott Williams and Wilkins",
number = "3",

}

TY - JOUR

T1 - A critical appraisal of the reporting of the National Acute Spinal Cord Injury Studies (II and III) of methylprednisolone in acute spinal cord injury

AU - Coleman, W. P.

AU - Benzel, E.

AU - Cahill, D. W.

AU - Ducker, T.

AU - Geisler, F.

AU - Green, B.

AU - Gropper, M. R.

AU - Goffin, J.

AU - Madsen, P. W.

AU - Maiman, D. J.

AU - Ondra, S. L.

AU - Rosner, M.

AU - Sasso, R. C.

AU - Trost, G. R.

AU - Zeidman, S.

PY - 2000

Y1 - 2000

N2 - From the beginning, the reporting of the results of National Acue Spinal Cord Injury. Studies (NASCIS) II and III has been incomplete, leaving clinicians in the spinal cord injury (SCI) community to use or avoid using methylprednisolone in acute SCI on the basis of faith rather than a publicly developed scientific consensus. NASCIS II was initially reported by National Institutes of Health announcements, National Institutes of Health facsimifiles to emergency room physicians, and the news media. The subsequent report in the New England Journal of Medicine implied that there was a positive result in the primary efficacy analysis for the entire 487 patient sample. However, this analysis was in fact negative, and the positive result was found only in a secondary analysis of the subgroup of patients who received treatment within 8 hours, In addition, that subgroup apparently had only 62 patients taking methylprednisolone and 67 receiving placebo. The NASCIS II and III reports embody specific choices of statistical methods that have strongly shaped the reporting of results but have not been adequately challenged or or even explained. These studies show statistical artifacts that call their results into question. In NASCIS II, the placebo group treated before 8 hours did poorly, not only when compared with the methylprednisolone group treated before 8 hours but even when compared with the placebo group treated after 8 hours. Thus, the positive result may have been caused by a weakness in the control group rather than any strength of methylprednisolone. In NASCIS III, a randomization imbalance occurred that allocated a disproportionate number of patients with no motor deficit (and therefore no chance for recovery) to the lower dose control group. When this imbalance is controlled for, much of the superiority of the higher dose group seems to disappear. The NASCIS group's decision to admit persons with minor SCIs with minimal or no motor deficit not only enables statistical artifacts it complicates the interpretation of results from the population actually sampled. Perhaps one half of the NASCIS III sample may have had at most a minor deficit. Thus, we do not know whether the results of these studies reflect the severely injured population to which they have been applied. The numbers, tables, and figures in the published reports are scant and are inconsistently defined, making it impossible even for professional statisticians to duplicate the analyses, to guess the effect of changes in assumptions, or to supply the missing parts of the picture. Nonetheless, even 9 years after NASCIS II, the primary, data have not been made public. The reporting of the NASCIS studies has fallen far short of the guidelines of the ICH/FDA and of the Evidence-based Medicine Group. Despite the lucrative "off label" markets for methylprednisolone in SCI, no Food and Drug Association indication has been obtained. There has been no public process of validation. These shortcomings have denied physicians the chance to use confidently a drug that many were enthusiastic about and has left them in an intolerably ambiguous position in their therapeutic choices, in their legal exposure, and in their ability to perform further research to help their patients.

AB - From the beginning, the reporting of the results of National Acue Spinal Cord Injury. Studies (NASCIS) II and III has been incomplete, leaving clinicians in the spinal cord injury (SCI) community to use or avoid using methylprednisolone in acute SCI on the basis of faith rather than a publicly developed scientific consensus. NASCIS II was initially reported by National Institutes of Health announcements, National Institutes of Health facsimifiles to emergency room physicians, and the news media. The subsequent report in the New England Journal of Medicine implied that there was a positive result in the primary efficacy analysis for the entire 487 patient sample. However, this analysis was in fact negative, and the positive result was found only in a secondary analysis of the subgroup of patients who received treatment within 8 hours, In addition, that subgroup apparently had only 62 patients taking methylprednisolone and 67 receiving placebo. The NASCIS II and III reports embody specific choices of statistical methods that have strongly shaped the reporting of results but have not been adequately challenged or or even explained. These studies show statistical artifacts that call their results into question. In NASCIS II, the placebo group treated before 8 hours did poorly, not only when compared with the methylprednisolone group treated before 8 hours but even when compared with the placebo group treated after 8 hours. Thus, the positive result may have been caused by a weakness in the control group rather than any strength of methylprednisolone. In NASCIS III, a randomization imbalance occurred that allocated a disproportionate number of patients with no motor deficit (and therefore no chance for recovery) to the lower dose control group. When this imbalance is controlled for, much of the superiority of the higher dose group seems to disappear. The NASCIS group's decision to admit persons with minor SCIs with minimal or no motor deficit not only enables statistical artifacts it complicates the interpretation of results from the population actually sampled. Perhaps one half of the NASCIS III sample may have had at most a minor deficit. Thus, we do not know whether the results of these studies reflect the severely injured population to which they have been applied. The numbers, tables, and figures in the published reports are scant and are inconsistently defined, making it impossible even for professional statisticians to duplicate the analyses, to guess the effect of changes in assumptions, or to supply the missing parts of the picture. Nonetheless, even 9 years after NASCIS II, the primary, data have not been made public. The reporting of the NASCIS studies has fallen far short of the guidelines of the ICH/FDA and of the Evidence-based Medicine Group. Despite the lucrative "off label" markets for methylprednisolone in SCI, no Food and Drug Association indication has been obtained. There has been no public process of validation. These shortcomings have denied physicians the chance to use confidently a drug that many were enthusiastic about and has left them in an intolerably ambiguous position in their therapeutic choices, in their legal exposure, and in their ability to perform further research to help their patients.

KW - Drug approval

KW - Methylprednisolone

KW - Spinal cord injufies

KW - U.S. Food and Drug Administration

UR - http://www.scopus.com/inward/record.url?scp=0034465202&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0034465202&partnerID=8YFLogxK

U2 - 10.1097/00002517-200006000-00001

DO - 10.1097/00002517-200006000-00001

M3 - Article

VL - 13

SP - 185

EP - 199

JO - Journal of Spinal Disorders

JF - Journal of Spinal Disorders

SN - 1536-0652

IS - 3

ER -