Deep learning-based fine-grained car make/model classification for visual surveillance

Erhan Gundogdu, Enes Sinan Parlldl, Berkan Solmaz, Veysel Yücesoy, Aykut Koç

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Fine-grained object recognition is a potential computer vision problem that has been recently addressed by utilizing deep Convolutional Neural Networks (CNNs). Nevertheless, the main disadvantage of classification methods relying on deep CNN models is the need for considerably large amount of data. In addition, there exists relatively less amount of annotated data for a real world application, such as the recognition of car models in a traffic surveillance system. To this end, we mainly concentrate on the classification of fine-grained car make and/or models for visual scenarios by the help of two different domains. First, a large-scale dataset including approximately 900K images is constructed from a website which includes fine-grained car models. According to their labels, a state-of-The-Art CNN model is trained on the constructed dataset. The second domain that is dealt with is the set of images collected from a camera integrated to a traffic surveillance system. These images, which are over 260K, are gathered by a special license plate detection method on top of a motion detection algorithm. An appropriately selected size of the image is cropped from the region of interest provided by the detected license plate location. These sets of images and their provided labels for more than 30 classes are employed to fine-Tune the CNN model which is already trained on the large scale dataset described above. To fine-Tune the network, the last two fully-connected layers are randomly initialized and the remaining layers are fine-Tuned in the second dataset. In this work, the transfer of a learned model on a large dataset to a smaller one has been successfully performed by utilizing both the limited annotated data of the traffic field and a large scale dataset with available annotations. Our experimental results both in the validation dataset and the real field show that the proposed methodology performs favorably against the training of the CNN model from scratch.

Original languageEnglish (US)
Title of host publicationCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies
EditorsYitzhak Yitzhaky, Robert James Stokes, Henri Bouma, Felicity Carlysle-Davies
PublisherSPIE
ISBN (Electronic)9781510613461
DOIs
StatePublished - Jan 1 2017
Externally publishedYes
EventCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies 2017 - Warsaw, Poland
Duration: Sep 11 2017Sep 12 2017

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume10441
ISSN (Print)0277-786X
ISSN (Electronic)1996-756X

Conference

ConferenceCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies 2017
CountryPoland
CityWarsaw
Period9/11/179/12/17

Fingerprint

Visual Surveillance
surveillance
learning
Railroad cars
Neural Network Model
Traffic
Neural networks
Surveillance
traffic
Motion Detection
Model
Labels
Object Recognition
Region of Interest
Real-world Applications
Large Data Sets
Computer Vision
Annotation
annotations
websites

Keywords

  • Deep convolutional neural networks
  • Fine-grained object recognition
  • Fine-Tuning
  • traffic surveillance

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

Gundogdu, E., Parlldl, E. S., Solmaz, B., Yücesoy, V., & Koç, A. (2017). Deep learning-based fine-grained car make/model classification for visual surveillance. In Y. Yitzhaky, R. J. Stokes, H. Bouma, & F. Carlysle-Davies (Eds.), Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies [104410J] (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 10441). SPIE. https://doi.org/10.1117/12.2278862

Deep learning-based fine-grained car make/model classification for visual surveillance. / Gundogdu, Erhan; Parlldl, Enes Sinan; Solmaz, Berkan; Yücesoy, Veysel; Koç, Aykut.

Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies. ed. / Yitzhak Yitzhaky; Robert James Stokes; Henri Bouma; Felicity Carlysle-Davies. SPIE, 2017. 104410J (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 10441).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Gundogdu, E, Parlldl, ES, Solmaz, B, Yücesoy, V & Koç, A 2017, Deep learning-based fine-grained car make/model classification for visual surveillance. in Y Yitzhaky, RJ Stokes, H Bouma & F Carlysle-Davies (eds), Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies., 104410J, Proceedings of SPIE - The International Society for Optical Engineering, vol. 10441, SPIE, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies 2017, Warsaw, Poland, 9/11/17. https://doi.org/10.1117/12.2278862
Gundogdu E, Parlldl ES, Solmaz B, Yücesoy V, Koç A. Deep learning-based fine-grained car make/model classification for visual surveillance. In Yitzhaky Y, Stokes RJ, Bouma H, Carlysle-Davies F, editors, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies. SPIE. 2017. 104410J. (Proceedings of SPIE - The International Society for Optical Engineering). https://doi.org/10.1117/12.2278862
Gundogdu, Erhan ; Parlldl, Enes Sinan ; Solmaz, Berkan ; Yücesoy, Veysel ; Koç, Aykut. / Deep learning-based fine-grained car make/model classification for visual surveillance. Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies. editor / Yitzhak Yitzhaky ; Robert James Stokes ; Henri Bouma ; Felicity Carlysle-Davies. SPIE, 2017. (Proceedings of SPIE - The International Society for Optical Engineering).
@inproceedings{fe7208d7f61c46258ac161ad6f439a72,
title = "Deep learning-based fine-grained car make/model classification for visual surveillance",
abstract = "Fine-grained object recognition is a potential computer vision problem that has been recently addressed by utilizing deep Convolutional Neural Networks (CNNs). Nevertheless, the main disadvantage of classification methods relying on deep CNN models is the need for considerably large amount of data. In addition, there exists relatively less amount of annotated data for a real world application, such as the recognition of car models in a traffic surveillance system. To this end, we mainly concentrate on the classification of fine-grained car make and/or models for visual scenarios by the help of two different domains. First, a large-scale dataset including approximately 900K images is constructed from a website which includes fine-grained car models. According to their labels, a state-of-The-Art CNN model is trained on the constructed dataset. The second domain that is dealt with is the set of images collected from a camera integrated to a traffic surveillance system. These images, which are over 260K, are gathered by a special license plate detection method on top of a motion detection algorithm. An appropriately selected size of the image is cropped from the region of interest provided by the detected license plate location. These sets of images and their provided labels for more than 30 classes are employed to fine-Tune the CNN model which is already trained on the large scale dataset described above. To fine-Tune the network, the last two fully-connected layers are randomly initialized and the remaining layers are fine-Tuned in the second dataset. In this work, the transfer of a learned model on a large dataset to a smaller one has been successfully performed by utilizing both the limited annotated data of the traffic field and a large scale dataset with available annotations. Our experimental results both in the validation dataset and the real field show that the proposed methodology performs favorably against the training of the CNN model from scratch.",
keywords = "Deep convolutional neural networks, Fine-grained object recognition, Fine-Tuning, traffic surveillance",
author = "Erhan Gundogdu and Parlldl, {Enes Sinan} and Berkan Solmaz and Veysel Y{\"u}cesoy and Aykut Ko{\cc}",
year = "2017",
month = "1",
day = "1",
doi = "10.1117/12.2278862",
language = "English (US)",
series = "Proceedings of SPIE - The International Society for Optical Engineering",
publisher = "SPIE",
editor = "Yitzhak Yitzhaky and Stokes, {Robert James} and Henri Bouma and Felicity Carlysle-Davies",
booktitle = "Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies",

}

TY - GEN

T1 - Deep learning-based fine-grained car make/model classification for visual surveillance

AU - Gundogdu, Erhan

AU - Parlldl, Enes Sinan

AU - Solmaz, Berkan

AU - Yücesoy, Veysel

AU - Koç, Aykut

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Fine-grained object recognition is a potential computer vision problem that has been recently addressed by utilizing deep Convolutional Neural Networks (CNNs). Nevertheless, the main disadvantage of classification methods relying on deep CNN models is the need for considerably large amount of data. In addition, there exists relatively less amount of annotated data for a real world application, such as the recognition of car models in a traffic surveillance system. To this end, we mainly concentrate on the classification of fine-grained car make and/or models for visual scenarios by the help of two different domains. First, a large-scale dataset including approximately 900K images is constructed from a website which includes fine-grained car models. According to their labels, a state-of-The-Art CNN model is trained on the constructed dataset. The second domain that is dealt with is the set of images collected from a camera integrated to a traffic surveillance system. These images, which are over 260K, are gathered by a special license plate detection method on top of a motion detection algorithm. An appropriately selected size of the image is cropped from the region of interest provided by the detected license plate location. These sets of images and their provided labels for more than 30 classes are employed to fine-Tune the CNN model which is already trained on the large scale dataset described above. To fine-Tune the network, the last two fully-connected layers are randomly initialized and the remaining layers are fine-Tuned in the second dataset. In this work, the transfer of a learned model on a large dataset to a smaller one has been successfully performed by utilizing both the limited annotated data of the traffic field and a large scale dataset with available annotations. Our experimental results both in the validation dataset and the real field show that the proposed methodology performs favorably against the training of the CNN model from scratch.

AB - Fine-grained object recognition is a potential computer vision problem that has been recently addressed by utilizing deep Convolutional Neural Networks (CNNs). Nevertheless, the main disadvantage of classification methods relying on deep CNN models is the need for considerably large amount of data. In addition, there exists relatively less amount of annotated data for a real world application, such as the recognition of car models in a traffic surveillance system. To this end, we mainly concentrate on the classification of fine-grained car make and/or models for visual scenarios by the help of two different domains. First, a large-scale dataset including approximately 900K images is constructed from a website which includes fine-grained car models. According to their labels, a state-of-The-Art CNN model is trained on the constructed dataset. The second domain that is dealt with is the set of images collected from a camera integrated to a traffic surveillance system. These images, which are over 260K, are gathered by a special license plate detection method on top of a motion detection algorithm. An appropriately selected size of the image is cropped from the region of interest provided by the detected license plate location. These sets of images and their provided labels for more than 30 classes are employed to fine-Tune the CNN model which is already trained on the large scale dataset described above. To fine-Tune the network, the last two fully-connected layers are randomly initialized and the remaining layers are fine-Tuned in the second dataset. In this work, the transfer of a learned model on a large dataset to a smaller one has been successfully performed by utilizing both the limited annotated data of the traffic field and a large scale dataset with available annotations. Our experimental results both in the validation dataset and the real field show that the proposed methodology performs favorably against the training of the CNN model from scratch.

KW - Deep convolutional neural networks

KW - Fine-grained object recognition

KW - Fine-Tuning

KW - traffic surveillance

UR - http://www.scopus.com/inward/record.url?scp=85038426240&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85038426240&partnerID=8YFLogxK

U2 - 10.1117/12.2278862

DO - 10.1117/12.2278862

M3 - Conference contribution

AN - SCOPUS:85038426240

T3 - Proceedings of SPIE - The International Society for Optical Engineering

BT - Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies

A2 - Yitzhaky, Yitzhak

A2 - Stokes, Robert James

A2 - Bouma, Henri

A2 - Carlysle-Davies, Felicity

PB - SPIE

ER -