Automatic Labeling of Vertebrae in Long-Length Intraoperative Imaging with a Multi-View, Region-Based CNN

Y. Huang, Craig Jones, X. Zhang, A. Johnston, Nafi Aygun, Timothy F Witham, P. A. Helm, J. H. Siewerdsen, Ali Uneri

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Purpose: A recent imaging method (viz., Long-Film) for capturing long-length images of the spine was enabled on the Oarm™ system. Proposed work uses a custom, multi-perspective, region-based convolutional neural network (R-CNN) for labeling vertebrae in Long-Film images and evaluates approaches for incorporating long contextual information to take advantage of the extended field-of-view and improve the labeling accuracy. Methods: Evaluated methods for incorporating contextual information include: (1) a recurrent network module with long short-term memory (LSTM) added after R-CNN classification; and (2) a post-processing, sequence-sorting step based on the label confidence scores. The models were trained and validated on 11,805 Long-Film images simulated from projections of 370 CT images and tested on 50 Long-Film images of 14 cadaveric specimens. Results: The multi-perspective R-CNN with LSTM module achieved 91.7% vertebrae level identification rate, compared to 72.4% when used without LSTM, thus demonstrating the improvement of incorporating contextual information. While sequence sorting achieved 89.4% in labeling accuracy, it failed to handle errors during detection and did not provide additional improvements when applied following the LSTM module. Conclusions: The proposed LSTM module significantly improved the labeling accuracy upon the base model through effective contextual information incorporation and training in an end-to-end fashion. Compared to sequence sorting, it showed more flexibility towards false positives and false negatives in vertebrae detection. The proposed model offers the potential to provide a valuable check for target localization and forms the basis for automatic measurement of spinal curvature changes in interventional settings.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2022
Subtitle of host publicationImage-Guided Procedures, Robotic Interventions, and Modeling
EditorsCristian A. Linte, Jeffrey H. Siewerdsen
ISBN (Electronic)9781510649439
StatePublished - 2022
EventMedical Imaging 2022: Image-Guided Procedures, Robotic Interventions, and Modeling - Virtual, Online
Duration: Mar 21 2022Mar 27 2022

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2022: Image-Guided Procedures, Robotic Interventions, and Modeling
CityVirtual, Online

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Automatic Labeling of Vertebrae in Long-Length Intraoperative Imaging with a Multi-View, Region-Based CNN'. Together they form a unique fingerprint.

Cite this