Computerized measurement of facial expression of emotions in schizophrenia

Christopher Alvino, Christian Kohler, Frederick Barrett, Raquel E. Gur, Ruben C. Gur, Ragini Verma

Research output: Contribution to journalArticle

Abstract

Deficits in the ability to express emotions characterize several neuropsychiatric disorders and are a hallmark of schizophrenia, and there is need for a method of quantifying expression, which is currently done by clinical ratings. This paper presents the development and validation of a computational framework for quantifying emotional expression differences between patients with schizophrenia and healthy controls. Each face is modeled as a combination of elastic regions, and expression changes are modeled as a deformation between a neutral face and an expressive face. Functions of these deformations, known as the regional volumetric difference (RVD) functions, form distinctive quantitative profiles of expressions. Employing pattern classification techniques, we have designed expression classifiers for the four universal emotions of happiness, sadness, anger and fear by training on RVD functions of expression changes. The classifiers were cross-validated and then applied to facial expression images of patients with schizophrenia and healthy controls. The classification score for each image reflects the extent to which the expressed emotion matches the intended emotion. Group-wise statistical analysis revealed this score to be significantly different between healthy controls and patients, especially in the case of anger. This score correlated with clinical severity of flat affect. These results encourage the use of such deformation based expression quantification measures for research in clinical applications that require the automated measurement of facial affect.

Original languageEnglish (US)
Pages (from-to)350-361
Number of pages12
JournalJournal of Neuroscience Methods
Volume163
Issue number2
DOIs
StatePublished - Jul 30 2007
Externally publishedYes

Fingerprint

Facial Expression
Schizophrenia
Emotions
Anger
Expressed Emotion
Happiness
Aptitude
Fear
Research

Keywords

  • Elastic deformations
  • Facial expressions quantification
  • Pattern classification
  • Regional volumetric differences
  • Support vector machines

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Computerized measurement of facial expression of emotions in schizophrenia. / Alvino, Christopher; Kohler, Christian; Barrett, Frederick; Gur, Raquel E.; Gur, Ruben C.; Verma, Ragini.

In: Journal of Neuroscience Methods, Vol. 163, No. 2, 30.07.2007, p. 350-361.

Research output: Contribution to journalArticle

Alvino, Christopher ; Kohler, Christian ; Barrett, Frederick ; Gur, Raquel E. ; Gur, Ruben C. ; Verma, Ragini. / Computerized measurement of facial expression of emotions in schizophrenia. In: Journal of Neuroscience Methods. 2007 ; Vol. 163, No. 2. pp. 350-361.
@article{b6c0af0a59a149a98d991ee4831212b3,
title = "Computerized measurement of facial expression of emotions in schizophrenia",
abstract = "Deficits in the ability to express emotions characterize several neuropsychiatric disorders and are a hallmark of schizophrenia, and there is need for a method of quantifying expression, which is currently done by clinical ratings. This paper presents the development and validation of a computational framework for quantifying emotional expression differences between patients with schizophrenia and healthy controls. Each face is modeled as a combination of elastic regions, and expression changes are modeled as a deformation between a neutral face and an expressive face. Functions of these deformations, known as the regional volumetric difference (RVD) functions, form distinctive quantitative profiles of expressions. Employing pattern classification techniques, we have designed expression classifiers for the four universal emotions of happiness, sadness, anger and fear by training on RVD functions of expression changes. The classifiers were cross-validated and then applied to facial expression images of patients with schizophrenia and healthy controls. The classification score for each image reflects the extent to which the expressed emotion matches the intended emotion. Group-wise statistical analysis revealed this score to be significantly different between healthy controls and patients, especially in the case of anger. This score correlated with clinical severity of flat affect. These results encourage the use of such deformation based expression quantification measures for research in clinical applications that require the automated measurement of facial affect.",
keywords = "Elastic deformations, Facial expressions quantification, Pattern classification, Regional volumetric differences, Support vector machines",
author = "Christopher Alvino and Christian Kohler and Frederick Barrett and Gur, {Raquel E.} and Gur, {Ruben C.} and Ragini Verma",
year = "2007",
month = "7",
day = "30",
doi = "10.1016/j.jneumeth.2007.03.002",
language = "English (US)",
volume = "163",
pages = "350--361",
journal = "Journal of Neuroscience Methods",
issn = "0165-0270",
publisher = "Elsevier",
number = "2",

}

TY - JOUR

T1 - Computerized measurement of facial expression of emotions in schizophrenia

AU - Alvino, Christopher

AU - Kohler, Christian

AU - Barrett, Frederick

AU - Gur, Raquel E.

AU - Gur, Ruben C.

AU - Verma, Ragini

PY - 2007/7/30

Y1 - 2007/7/30

N2 - Deficits in the ability to express emotions characterize several neuropsychiatric disorders and are a hallmark of schizophrenia, and there is need for a method of quantifying expression, which is currently done by clinical ratings. This paper presents the development and validation of a computational framework for quantifying emotional expression differences between patients with schizophrenia and healthy controls. Each face is modeled as a combination of elastic regions, and expression changes are modeled as a deformation between a neutral face and an expressive face. Functions of these deformations, known as the regional volumetric difference (RVD) functions, form distinctive quantitative profiles of expressions. Employing pattern classification techniques, we have designed expression classifiers for the four universal emotions of happiness, sadness, anger and fear by training on RVD functions of expression changes. The classifiers were cross-validated and then applied to facial expression images of patients with schizophrenia and healthy controls. The classification score for each image reflects the extent to which the expressed emotion matches the intended emotion. Group-wise statistical analysis revealed this score to be significantly different between healthy controls and patients, especially in the case of anger. This score correlated with clinical severity of flat affect. These results encourage the use of such deformation based expression quantification measures for research in clinical applications that require the automated measurement of facial affect.

AB - Deficits in the ability to express emotions characterize several neuropsychiatric disorders and are a hallmark of schizophrenia, and there is need for a method of quantifying expression, which is currently done by clinical ratings. This paper presents the development and validation of a computational framework for quantifying emotional expression differences between patients with schizophrenia and healthy controls. Each face is modeled as a combination of elastic regions, and expression changes are modeled as a deformation between a neutral face and an expressive face. Functions of these deformations, known as the regional volumetric difference (RVD) functions, form distinctive quantitative profiles of expressions. Employing pattern classification techniques, we have designed expression classifiers for the four universal emotions of happiness, sadness, anger and fear by training on RVD functions of expression changes. The classifiers were cross-validated and then applied to facial expression images of patients with schizophrenia and healthy controls. The classification score for each image reflects the extent to which the expressed emotion matches the intended emotion. Group-wise statistical analysis revealed this score to be significantly different between healthy controls and patients, especially in the case of anger. This score correlated with clinical severity of flat affect. These results encourage the use of such deformation based expression quantification measures for research in clinical applications that require the automated measurement of facial affect.

KW - Elastic deformations

KW - Facial expressions quantification

KW - Pattern classification

KW - Regional volumetric differences

KW - Support vector machines

UR - http://www.scopus.com/inward/record.url?scp=34249977906&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=34249977906&partnerID=8YFLogxK

U2 - 10.1016/j.jneumeth.2007.03.002

DO - 10.1016/j.jneumeth.2007.03.002

M3 - Article

C2 - 17442398

AN - SCOPUS:34249977906

VL - 163

SP - 350

EP - 361

JO - Journal of Neuroscience Methods

JF - Journal of Neuroscience Methods

SN - 0165-0270

IS - 2

ER -