The beta error and sample size determination in clinical trials in emergency medicine

In the analysis of a clinical trial an investigator may fail to discern a statistically significant difference in outcome between control and experimental groups, when in fact one exists. Failure to demonstrate such a difference when it actually exists is known as "type II" error, and its probability of occurring is termed "beta". The purpose of our study was to determine the distribution of beta errors in negative trials in the Journal of the American College of Emergency Physicians (JACEP) (1972-1979) and Annals of Emergency Medicine (1980-1984). All negative comparative clinical trials appearing in JACEP and Annals from volume 1 (1972) to volume 13 (1984) were surveyed and were eligible for inclusion in the study. A trial was defined as negative if the investigator specifically stated that there was no significant difference in outcome between the experimental and control groups. For each negative trial the following parameters were calculated: beta error, based on the sample size used and the difference determined to be important to detect clinically; sample size required to detect a clinically meaningful difference as determined by the authors of this study; and minimum true difference that had to be detected in the trial at a beta equal to 0.20, to discern a statistically significant result. For the 13 years surveyed, we found 21 endpoints in 14 negative trials that were analyzable. Only one of the trials (7.1%) addressed the issues of beta errors and sample size determination. In the remaining 13 negative trials, the calculated beta error ranged from .60 to .97. For the endpoints analyzed, a sample size of up to 450 times larger than that used would have been required to detect a clinically important difference. The minimal difference that could have been detected to discern a statistically significant difference for the endpoints studied was greater than the difference estimated to be important to detect clinically in all studies when beta was set at .20, given the sample size used. These results indicate that due to the small sample sizes used in the investigations analyzed, there was a high probability of missing a clinically meaningful difference if one existed.