Scene parsing using a prior world model

Gregory D. Hager, Ben Wegbreit

Research output: Contribution to journalArticle


We present a new paradigm for constructing a 3D model of a scene from images. Our approach makes strong use of a prior 3D model of the scene. Changes from scene to scene are regarded as a Markov dynamical system, which is described by a probabilistic transition model. From the prior 3D scene model, the model of scene change dynamics, and a newly acquired image, we compute the new 3D scene model which is most consistent with the observed image and the changes from the prior model. The use of a prior 3D scene model allows the method to deal with complex scenes, maintain hidden state, respect object persistence, perform object segmentation, and provides computational efficiencies. In this paper we formalize a mathematical framework for physically consistent 3D scene models, and changes to scene models that preserve physical consistency. From this framework, we first derive a generic scene model optimization algorithm for the general 3D scene interpretation problem, and we then present a polynomial time approximation for this algorithm. We detail the implementation of the algorithm for range images computed by stereo imaging, and present extensive experimental results on sequences of scenes containing dozens of objects and multiple changes from scene to scene.

Original languageEnglish (US)
Pages (from-to)1477-1507
Number of pages31
JournalInternational Journal of Robotics Research
Issue number12
StatePublished - Oct 1 2011


  • 3D scene models
  • Range sensing
  • recognition
  • scene interpretation
  • sensing and perception computer vision

ASJC Scopus subject areas

  • Software
  • Modeling and Simulation
  • Mechanical Engineering
  • Artificial Intelligence
  • Electrical and Electronic Engineering
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Scene parsing using a prior world model'. Together they form a unique fingerprint.

Cite this