The evaluation of research methods during disaster exercises: Applicability for improving disaster health management

Introduction: The objective of this study was to investigate whether disaster exercises can be used as a proxy environment to evaluate potential research instruments designed to study the application of medical care management resources during a disaster. Methods: During an 06 April 2005 Ministerial-level exercise in the Netherlands, three functional areas of patient contact were assessed: (1) Command and Control, through the application of an existing incident management system questionnaire; (2) patient flow and quality of patient distribution, through registration of data from prehospital casualty collection points, ambulances, and participating trauma centers (with inclusion of data in a flow chart); and (3) hospital coping capacity, through timed registration reports from participating trauma centers. Results: The existing incident management system questionnaire used for evaluating Command and Control during a disaster exercise would benefit from minor adaptations and validation that could not be anticipated in the exercise planning stage. Patient flow and the quality of patient distribution could not be studied during the exercise because of inconsistencies among data, and lack of data from various collection points. Coping capacity was better measured by using 10-minute rather than one hour time intervals, but provided little information regarding bottlenecks in surge capacity. Conclusion: Research instruments can be evaluated and improved when tested during a disaster exercise. Lack of data recovery hampers disaster research even in the artificial setting of a national disaster exercise. Providers at every level must be aware that proper data collection is essential to improve the quality of health care during a disaster, and that predisaster cooperation is crucial to validate patient outcomes. These problems must be addressed pre-exercise by stakeholders and decision-makers during planning, education, and training. If not, disaster exercises will not meet their full potential.