Shadow casting out of plane (SCOOP) candidates for human and vehicle detection in aerial imagery

Vladimir Reilly, Berkan Solmaz, Mubarak Shah

Research output: Contribution to journalArticle

Abstract

In this paper, we propose a method for detecting humans and vehicles in imagery taken from a UAV. This is a challenging problem due to a limited number of pixels on target, which makes it more difficult to distinguish objects from background clutter, and results in much larger search space. We propose a method for constraining the search based on a number of geometric constraints obtained from the metadata. Specifically, we obtain the orientation of ground plane normal, the orientation of shadows cast by out of plane objects in the scene, and the relationship between object heights and the size of their corresponding shadows. We use the aforementioned information in a geometry-based shadow, and ground-plane normal blob detector, which provides an initial estimation for locations of shadow casting out of plane (SCOOP) objects in the scene. These SCOOP candidate locations are then classified as either human or clutter using a combination of wavelet features and a Support Vector Machine. To detect vehicles, we similarly find potential vehicle candidates by combining SCOOP and inverted-SCOOP candidates and then classify them using wavelet features and SVM. Our method works on a single frame, and unlike motion detection based methods, it bypasses the entire pipeline of registration, motion detection, and tracking. This method allows for detection of stationary and slowly moving humans and vehicles while avoiding the search across the entire image, allowing accurate and fast localization. We show impressive results on sequences from VIVID and CLIF datasets and provide comparative analysis.

Original languageEnglish (US)
Pages (from-to)350-366
Number of pages17
JournalInternational Journal of Computer Vision
Volume101
Issue number2
DOIs
StatePublished - Jan 1 2013
Externally publishedYes

Fingerprint

Casting
Antennas
Unmanned aerial vehicles (UAV)
Metadata
Support vector machines
Pipelines
Pixels
Detectors
Geometry

Keywords

  • Aerial surveillance
  • Human detection
  • Metadata
  • Shadow
  • UAV
  • Vehicle detection

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Shadow casting out of plane (SCOOP) candidates for human and vehicle detection in aerial imagery. / Reilly, Vladimir; Solmaz, Berkan; Shah, Mubarak.

In: International Journal of Computer Vision, Vol. 101, No. 2, 01.01.2013, p. 350-366.

Research output: Contribution to journalArticle

@article{f7368edbce9c4144b347a7610526725a,
title = "Shadow casting out of plane (SCOOP) candidates for human and vehicle detection in aerial imagery",
abstract = "In this paper, we propose a method for detecting humans and vehicles in imagery taken from a UAV. This is a challenging problem due to a limited number of pixels on target, which makes it more difficult to distinguish objects from background clutter, and results in much larger search space. We propose a method for constraining the search based on a number of geometric constraints obtained from the metadata. Specifically, we obtain the orientation of ground plane normal, the orientation of shadows cast by out of plane objects in the scene, and the relationship between object heights and the size of their corresponding shadows. We use the aforementioned information in a geometry-based shadow, and ground-plane normal blob detector, which provides an initial estimation for locations of shadow casting out of plane (SCOOP) objects in the scene. These SCOOP candidate locations are then classified as either human or clutter using a combination of wavelet features and a Support Vector Machine. To detect vehicles, we similarly find potential vehicle candidates by combining SCOOP and inverted-SCOOP candidates and then classify them using wavelet features and SVM. Our method works on a single frame, and unlike motion detection based methods, it bypasses the entire pipeline of registration, motion detection, and tracking. This method allows for detection of stationary and slowly moving humans and vehicles while avoiding the search across the entire image, allowing accurate and fast localization. We show impressive results on sequences from VIVID and CLIF datasets and provide comparative analysis.",
keywords = "Aerial surveillance, Human detection, Metadata, Shadow, UAV, Vehicle detection",
author = "Vladimir Reilly and Berkan Solmaz and Mubarak Shah",
year = "2013",
month = "1",
day = "1",
doi = "10.1007/s11263-012-0580-1",
language = "English (US)",
volume = "101",
pages = "350--366",
journal = "International Journal of Computer Vision",
issn = "0920-5691",
publisher = "Springer Netherlands",
number = "2",

}

TY - JOUR

T1 - Shadow casting out of plane (SCOOP) candidates for human and vehicle detection in aerial imagery

AU - Reilly, Vladimir

AU - Solmaz, Berkan

AU - Shah, Mubarak

PY - 2013/1/1

Y1 - 2013/1/1

N2 - In this paper, we propose a method for detecting humans and vehicles in imagery taken from a UAV. This is a challenging problem due to a limited number of pixels on target, which makes it more difficult to distinguish objects from background clutter, and results in much larger search space. We propose a method for constraining the search based on a number of geometric constraints obtained from the metadata. Specifically, we obtain the orientation of ground plane normal, the orientation of shadows cast by out of plane objects in the scene, and the relationship between object heights and the size of their corresponding shadows. We use the aforementioned information in a geometry-based shadow, and ground-plane normal blob detector, which provides an initial estimation for locations of shadow casting out of plane (SCOOP) objects in the scene. These SCOOP candidate locations are then classified as either human or clutter using a combination of wavelet features and a Support Vector Machine. To detect vehicles, we similarly find potential vehicle candidates by combining SCOOP and inverted-SCOOP candidates and then classify them using wavelet features and SVM. Our method works on a single frame, and unlike motion detection based methods, it bypasses the entire pipeline of registration, motion detection, and tracking. This method allows for detection of stationary and slowly moving humans and vehicles while avoiding the search across the entire image, allowing accurate and fast localization. We show impressive results on sequences from VIVID and CLIF datasets and provide comparative analysis.

AB - In this paper, we propose a method for detecting humans and vehicles in imagery taken from a UAV. This is a challenging problem due to a limited number of pixels on target, which makes it more difficult to distinguish objects from background clutter, and results in much larger search space. We propose a method for constraining the search based on a number of geometric constraints obtained from the metadata. Specifically, we obtain the orientation of ground plane normal, the orientation of shadows cast by out of plane objects in the scene, and the relationship between object heights and the size of their corresponding shadows. We use the aforementioned information in a geometry-based shadow, and ground-plane normal blob detector, which provides an initial estimation for locations of shadow casting out of plane (SCOOP) objects in the scene. These SCOOP candidate locations are then classified as either human or clutter using a combination of wavelet features and a Support Vector Machine. To detect vehicles, we similarly find potential vehicle candidates by combining SCOOP and inverted-SCOOP candidates and then classify them using wavelet features and SVM. Our method works on a single frame, and unlike motion detection based methods, it bypasses the entire pipeline of registration, motion detection, and tracking. This method allows for detection of stationary and slowly moving humans and vehicles while avoiding the search across the entire image, allowing accurate and fast localization. We show impressive results on sequences from VIVID and CLIF datasets and provide comparative analysis.

KW - Aerial surveillance

KW - Human detection

KW - Metadata

KW - Shadow

KW - UAV

KW - Vehicle detection

UR - http://www.scopus.com/inward/record.url?scp=84873199492&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84873199492&partnerID=8YFLogxK

U2 - 10.1007/s11263-012-0580-1

DO - 10.1007/s11263-012-0580-1

M3 - Article

AN - SCOPUS:84873199492

VL - 101

SP - 350

EP - 366

JO - International Journal of Computer Vision

JF - International Journal of Computer Vision

SN - 0920-5691

IS - 2

ER -