Examining the Relative Utility of PBIS Implementation Fidelity Scores in Relation to Student Outcomes

Elise Touris Pas, Stacy R. Johnson, Katrina J. Debnam, Chris S. Hulleman, Catherine P. Bradshaw

Research output: Contribution to journalArticle

Abstract

There are several widely used Positive Behavioral Interventions and Supports (PBIS) implementation fidelity measures, including the Schoolwide Evaluation Tool (SET), but there is limited empirical investigation substantiating the association between specific fidelity cut point scores and student outcomes. This article examined different SET cut points in reference to school-level outcomes across 180 elementary, middle, and high schools participating in randomized trials of PBIS. Specifically, we examined the implementation fidelity achieved in the sample, as well as within each school level (i.e., elementary, middle, and high). Using the binary complier index, we examined the association of each of three SET cut points with behavioral and academic outcomes using t tests and receiver operating characteristic (ROC) curve analyses. Although the t tests did not provide confirmation of one specific cut point, post hoc analyses suggested that the cut point may need to be tailored to school levels. There was moderate convergence between implementation on one scale (i.e., responses to behavioral violations) and academic achievement, based on ROC curve analyses for the 70% and 80% cut points. Implications for research and practice are discussed.

Original languageEnglish (US)
Pages (from-to)6-15
Number of pages10
JournalRemedial and Special Education
Volume40
Issue number1
DOIs
StatePublished - Feb 1 2019

Fingerprint

Students
ROC Curve
school
recipient
student
evaluation
academic achievement
Research

Keywords

  • cut point scores
  • fidelity
  • positive behavior supports
  • randomized controlled trial

ASJC Scopus subject areas

  • Education
  • Public Health, Environmental and Occupational Health

Cite this

Examining the Relative Utility of PBIS Implementation Fidelity Scores in Relation to Student Outcomes. / Pas, Elise Touris; Johnson, Stacy R.; Debnam, Katrina J.; Hulleman, Chris S.; Bradshaw, Catherine P.

In: Remedial and Special Education, Vol. 40, No. 1, 01.02.2019, p. 6-15.

Research output: Contribution to journalArticle

Pas, Elise Touris ; Johnson, Stacy R. ; Debnam, Katrina J. ; Hulleman, Chris S. ; Bradshaw, Catherine P. / Examining the Relative Utility of PBIS Implementation Fidelity Scores in Relation to Student Outcomes. In: Remedial and Special Education. 2019 ; Vol. 40, No. 1. pp. 6-15.
@article{0c38669936704081b7305a3b9f161c1c,
title = "Examining the Relative Utility of PBIS Implementation Fidelity Scores in Relation to Student Outcomes",
abstract = "There are several widely used Positive Behavioral Interventions and Supports (PBIS) implementation fidelity measures, including the Schoolwide Evaluation Tool (SET), but there is limited empirical investigation substantiating the association between specific fidelity cut point scores and student outcomes. This article examined different SET cut points in reference to school-level outcomes across 180 elementary, middle, and high schools participating in randomized trials of PBIS. Specifically, we examined the implementation fidelity achieved in the sample, as well as within each school level (i.e., elementary, middle, and high). Using the binary complier index, we examined the association of each of three SET cut points with behavioral and academic outcomes using t tests and receiver operating characteristic (ROC) curve analyses. Although the t tests did not provide confirmation of one specific cut point, post hoc analyses suggested that the cut point may need to be tailored to school levels. There was moderate convergence between implementation on one scale (i.e., responses to behavioral violations) and academic achievement, based on ROC curve analyses for the 70{\%} and 80{\%} cut points. Implications for research and practice are discussed.",
keywords = "cut point scores, fidelity, positive behavior supports, randomized controlled trial",
author = "Pas, {Elise Touris} and Johnson, {Stacy R.} and Debnam, {Katrina J.} and Hulleman, {Chris S.} and Bradshaw, {Catherine P.}",
year = "2019",
month = "2",
day = "1",
doi = "10.1177/0741932518805192",
language = "English (US)",
volume = "40",
pages = "6--15",
journal = "Remedial and Special Education",
issn = "0741-9325",
publisher = "SAGE Publications Inc.",
number = "1",

}

TY - JOUR

T1 - Examining the Relative Utility of PBIS Implementation Fidelity Scores in Relation to Student Outcomes

AU - Pas, Elise Touris

AU - Johnson, Stacy R.

AU - Debnam, Katrina J.

AU - Hulleman, Chris S.

AU - Bradshaw, Catherine P.

PY - 2019/2/1

Y1 - 2019/2/1

N2 - There are several widely used Positive Behavioral Interventions and Supports (PBIS) implementation fidelity measures, including the Schoolwide Evaluation Tool (SET), but there is limited empirical investigation substantiating the association between specific fidelity cut point scores and student outcomes. This article examined different SET cut points in reference to school-level outcomes across 180 elementary, middle, and high schools participating in randomized trials of PBIS. Specifically, we examined the implementation fidelity achieved in the sample, as well as within each school level (i.e., elementary, middle, and high). Using the binary complier index, we examined the association of each of three SET cut points with behavioral and academic outcomes using t tests and receiver operating characteristic (ROC) curve analyses. Although the t tests did not provide confirmation of one specific cut point, post hoc analyses suggested that the cut point may need to be tailored to school levels. There was moderate convergence between implementation on one scale (i.e., responses to behavioral violations) and academic achievement, based on ROC curve analyses for the 70% and 80% cut points. Implications for research and practice are discussed.

AB - There are several widely used Positive Behavioral Interventions and Supports (PBIS) implementation fidelity measures, including the Schoolwide Evaluation Tool (SET), but there is limited empirical investigation substantiating the association between specific fidelity cut point scores and student outcomes. This article examined different SET cut points in reference to school-level outcomes across 180 elementary, middle, and high schools participating in randomized trials of PBIS. Specifically, we examined the implementation fidelity achieved in the sample, as well as within each school level (i.e., elementary, middle, and high). Using the binary complier index, we examined the association of each of three SET cut points with behavioral and academic outcomes using t tests and receiver operating characteristic (ROC) curve analyses. Although the t tests did not provide confirmation of one specific cut point, post hoc analyses suggested that the cut point may need to be tailored to school levels. There was moderate convergence between implementation on one scale (i.e., responses to behavioral violations) and academic achievement, based on ROC curve analyses for the 70% and 80% cut points. Implications for research and practice are discussed.

KW - cut point scores

KW - fidelity

KW - positive behavior supports

KW - randomized controlled trial

UR - http://www.scopus.com/inward/record.url?scp=85060556488&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85060556488&partnerID=8YFLogxK

U2 - 10.1177/0741932518805192

DO - 10.1177/0741932518805192

M3 - Article

AN - SCOPUS:85060556488

VL - 40

SP - 6

EP - 15

JO - Remedial and Special Education

JF - Remedial and Special Education

SN - 0741-9325

IS - 1

ER -