Scene parsing using a prior world model

Gregory Hager, Ben Wegbreit

Research output: Contribution to journalArticle

Abstract

We present a new paradigm for constructing a 3D model of a scene from images. Our approach makes strong use of a prior 3D model of the scene. Changes from scene to scene are regarded as a Markov dynamical system, which is described by a probabilistic transition model. From the prior 3D scene model, the model of scene change dynamics, and a newly acquired image, we compute the new 3D scene model which is most consistent with the observed image and the changes from the prior model. The use of a prior 3D scene model allows the method to deal with complex scenes, maintain hidden state, respect object persistence, perform object segmentation, and provides computational efficiencies. In this paper we formalize a mathematical framework for physically consistent 3D scene models, and changes to scene models that preserve physical consistency. From this framework, we first derive a generic scene model optimization algorithm for the general 3D scene interpretation problem, and we then present a polynomial time approximation for this algorithm. We detail the implementation of the algorithm for range images computed by stereo imaging, and present extensive experimental results on sequences of scenes containing dozens of objects and multiple changes from scene to scene.

Original languageEnglish (US)
Pages (from-to)1477-1507
Number of pages31
JournalInternational Journal of Robotics Research
Volume30
Issue number12
DOIs
StatePublished - Oct 2011

Fingerprint

Parsing
3D Model
Model
Range Image
Transition Model
Probabilistic Model
Computational Efficiency
Persistence
Optimization Algorithm
Polynomial time
Segmentation
Dynamical system
Paradigm
Computational efficiency
Imaging
Dynamical systems
Experimental Results
Polynomials
Approximation
Imaging techniques

Keywords

  • 3D scene models
  • Range sensing
  • recognition
  • scene interpretation
  • sensing and perception computer vision

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Mechanical Engineering
  • Software
  • Artificial Intelligence
  • Applied Mathematics
  • Modeling and Simulation

Cite this

Scene parsing using a prior world model. / Hager, Gregory; Wegbreit, Ben.

In: International Journal of Robotics Research, Vol. 30, No. 12, 10.2011, p. 1477-1507.

Research output: Contribution to journalArticle

Hager, Gregory ; Wegbreit, Ben. / Scene parsing using a prior world model. In: International Journal of Robotics Research. 2011 ; Vol. 30, No. 12. pp. 1477-1507.
@article{683235191bfd4180b5993e29c033c148,
title = "Scene parsing using a prior world model",
abstract = "We present a new paradigm for constructing a 3D model of a scene from images. Our approach makes strong use of a prior 3D model of the scene. Changes from scene to scene are regarded as a Markov dynamical system, which is described by a probabilistic transition model. From the prior 3D scene model, the model of scene change dynamics, and a newly acquired image, we compute the new 3D scene model which is most consistent with the observed image and the changes from the prior model. The use of a prior 3D scene model allows the method to deal with complex scenes, maintain hidden state, respect object persistence, perform object segmentation, and provides computational efficiencies. In this paper we formalize a mathematical framework for physically consistent 3D scene models, and changes to scene models that preserve physical consistency. From this framework, we first derive a generic scene model optimization algorithm for the general 3D scene interpretation problem, and we then present a polynomial time approximation for this algorithm. We detail the implementation of the algorithm for range images computed by stereo imaging, and present extensive experimental results on sequences of scenes containing dozens of objects and multiple changes from scene to scene.",
keywords = "3D scene models, Range sensing, recognition, scene interpretation, sensing and perception computer vision",
author = "Gregory Hager and Ben Wegbreit",
year = "2011",
month = "10",
doi = "10.1177/0278364911399340",
language = "English (US)",
volume = "30",
pages = "1477--1507",
journal = "International Journal of Robotics Research",
issn = "0278-3649",
publisher = "SAGE Publications Inc.",
number = "12",

}

TY - JOUR

T1 - Scene parsing using a prior world model

AU - Hager, Gregory

AU - Wegbreit, Ben

PY - 2011/10

Y1 - 2011/10

N2 - We present a new paradigm for constructing a 3D model of a scene from images. Our approach makes strong use of a prior 3D model of the scene. Changes from scene to scene are regarded as a Markov dynamical system, which is described by a probabilistic transition model. From the prior 3D scene model, the model of scene change dynamics, and a newly acquired image, we compute the new 3D scene model which is most consistent with the observed image and the changes from the prior model. The use of a prior 3D scene model allows the method to deal with complex scenes, maintain hidden state, respect object persistence, perform object segmentation, and provides computational efficiencies. In this paper we formalize a mathematical framework for physically consistent 3D scene models, and changes to scene models that preserve physical consistency. From this framework, we first derive a generic scene model optimization algorithm for the general 3D scene interpretation problem, and we then present a polynomial time approximation for this algorithm. We detail the implementation of the algorithm for range images computed by stereo imaging, and present extensive experimental results on sequences of scenes containing dozens of objects and multiple changes from scene to scene.

AB - We present a new paradigm for constructing a 3D model of a scene from images. Our approach makes strong use of a prior 3D model of the scene. Changes from scene to scene are regarded as a Markov dynamical system, which is described by a probabilistic transition model. From the prior 3D scene model, the model of scene change dynamics, and a newly acquired image, we compute the new 3D scene model which is most consistent with the observed image and the changes from the prior model. The use of a prior 3D scene model allows the method to deal with complex scenes, maintain hidden state, respect object persistence, perform object segmentation, and provides computational efficiencies. In this paper we formalize a mathematical framework for physically consistent 3D scene models, and changes to scene models that preserve physical consistency. From this framework, we first derive a generic scene model optimization algorithm for the general 3D scene interpretation problem, and we then present a polynomial time approximation for this algorithm. We detail the implementation of the algorithm for range images computed by stereo imaging, and present extensive experimental results on sequences of scenes containing dozens of objects and multiple changes from scene to scene.

KW - 3D scene models

KW - Range sensing

KW - recognition

KW - scene interpretation

KW - sensing and perception computer vision

UR - http://www.scopus.com/inward/record.url?scp=80054765294&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80054765294&partnerID=8YFLogxK

U2 - 10.1177/0278364911399340

DO - 10.1177/0278364911399340

M3 - Article

AN - SCOPUS:80054765294

VL - 30

SP - 1477

EP - 1507

JO - International Journal of Robotics Research

JF - International Journal of Robotics Research

SN - 0278-3649

IS - 12

ER -