Monitoring rare serious adverse events from a new treatment and testing for a difference from historical controls

Background: We detail the design of a study to monitor the safety of including albendazole to an existing treatment regimen to eliminate lymphatic filariasis. We wish to show that this new regimen does not increase the rate of a rare serious adverse event (SAE) compared to the old regimen. Controlled but small clinical trials have not detected any increase in the SAE using albendazole, and it is known to have added benefits; therefore, it is unethical to randomize patients to the old regimen. Purpose: A sample size for the new regimen is needed to test that the new rate of SAE is noninferior to the historic rate. If the new regimen does have an inferior rate of SAE then we wish to stop the study early. This setup is different from traditional early stopping for efficacy and futility. In that traditional case, the two stopping decisions are relative to the same null hypothesis of equality, while in our setup, we have two different null hypotheses: the noninferiority null and the equality null. When testing the former, we need not stop early if the new regimen appears better because no subjects are receiving the old regimen anymore anyway. When testing the equality of SAE rates, however, we want to stop early if the new regimen has a significantly higher rate of SAE. Methods: We create a design that uses an exact difference in proportions test for testing noninferiority, but calculates maximal sample size based on conditional power which treats the historical rates as true rates. The design allows for early stopping if the new treatment appears inferior with respect to SAE rate but makes no corrections for multiple testing. We explore the properties of this naive design without assuming the historical rates are known. Results: For our example, we show that our naive design strategy bounds the type I error of the noninferiority hypothesis in all cases and bounds it for the equality hypothesis at 0.05, as long as the true SAE rate is <0.00015. The same design has unconditional power for the noninferiority hypothesis greater than the nominal 80% as long as the true SAE rate for both regimens are <0.00025. Limitations: The type I and power results above hold only for our historical sample size of 17 877. We expect similar type I and power properties to hold with studies with SAE rates similar or less (i.e., < 0.00015) and historical sample sizes similar or smaller. Conclusions: Our design for comparing very rare historical SAE rates to SAE rates of a new treatment has large power to conclude noninferiority of the new treatment SAE rate when both rates are equal, but allows early stopping if the new SAE rates are worse.