A system for continual quality improvement of normal tissue delineation for radiation therapy treatment planning

Purpose: To implement the "plan-do-check-act" (PDCA) cycle for the continual quality improvement of normal tissue contours used for radiation therapy treatment planning. Methods and Materials: The CT scans of patients treated for tumors of the brain, head and neck, thorax, pancreas and prostate were selected for this study. For each scan, a radiation oncologist and a diagnostic radiologist, outlined the normal tissues ("gold" contours) using Radiation Therapy Oncology Group (RTOG) guidelines. A total of 30 organs were delineated. Independently, 5 board-certified dosimetrists and 1 trainee then outlined the same organs. Metrics used to compare the agreement between the dosimetrists' contours and the gold contours included the Dice Similarity Coefficient (DSC), and a penalty function using distance to agreement. Based on these scores, dosimetrists were re-trained on those organs in which they did not receive a passing score, and they were subsequently re-tested. Results: Passing scores were achieved on 19 of 30 organs evaluated. These scores were correlated to organ volume. For organ volumes <8 cc, the average DSC was 0.61 vs organ volumes ≥8 cc, for which the average DSC was 0.91 (P=.005). Normal tissues that had the lowest scores included the lenses, optic nerves, chiasm, cochlea, and esophagus. Of the 11 organs that were considered for re-testing, 10 showed improvement in the average score, and statistically significant improvement was noted in more than half of these organs after education and re-assessment. Conclusions: The results of this study indicate the feasibility of applying the PDCA cycle to assess competence in the delineation of individual organs, and to identify areas for improvement. With testing, guidance, and re-evaluation, contouring consistency can be obtained across multiple dosimetrists. Our expectation is that continual quality improvement using the PDCA approach will ensure more accurate treatments and dose assessment in radiotherapy treatment planning and delivery.