Alarm fatigue in intensive care units (ICU) is one of the top healthcare issues in the US. False alarms in ICU will decrease the quality of care and staff response time over the alarms. Normally, false alarm will cause desensitization of the clinical staff which leads to warnings and misleading, if the triggered alarm is true. In this study, we have proposed a multi-model ensemble approach to reduce the false alarm rate in monitoring systems. We have used 750 patient records from PhysioNet database. At First arrhythmia based features from electrocardiogram (ECG), arterial blood pressure (ABP) and photoplethysmogram (PPG) features were extracted from the records. Next, the dataset has been separated into two subsets on the basis of available features information. The first dataset (DS1) is the combination of ECG physiological, ABP and PPG features. Their correlation coefficient and p-values criteria have been applied for relevant alarm-wise feature-set selection, and random forest classifier was used for model development and validation. The threshold based approach was used on second dataset (DS2) which is the combination of arrhythmia, ABP and PPG features. The developed ensemble model is able to achieve sensitivity 83.33-100 % (average 95.56 %) being true alarms and suppress false alarms rate 66.67-89% (average 77.25%). The predictability of classifier shows the advantage to deal with unbalanced set of information, therefore overall model performance has reached to 83.96% accuracy.