Reliability of the American Academy of Sleep Medicine Rules for Assessing Sleep Depth in Clinical Practice

Magdy Younes, Samuel T. Kuna, Allan I. Pack, James K. Walsh, Clete A. Kushida, Bethany Staley, Grace W. Pien

Research output: Contribution to journalArticle

Abstract

Study Objectives: The American Academy of Sleep Medicine has published manuals for scoring polysomnograms that recommend time spent in non-rapid eye movement sleep stages (stage N1, N2, and N3 sleep) be reported. Given the well-established large interrater variability in scoring stage N1 and N3 sleep, we determined the range of time in stage N1 and N3 sleep scored by a large number of technologists when compared to reasonably estimated true values. Methods: Polysomnograms of 70 females were scored by 10 highly trained sleep technologists, two each from five different academic sleep laboratories. Range and confidence interval (CI = difference between the 5th and 95th percentiles) of the 10 times spent in stage N1 and N3 sleep assigned in each polysomnogram were determined. Average values of times spent in stage N1 and N3 sleep generated by the 10 technologists in each polysomnogram were considered representative of the true values for the individual polysomnogram. Accuracy of different technologists in estimating delta wave duration was determined by comparing their scores to digitally determined durations. Results: The CI range of the ten N1 scores was 4 to 39 percent of total sleep time (% TST) in different polysomnograms (mean CI ± standard deviation = 11.1 ± 7.1 % TST). Corresponding range for N3 was 1 to 28 % TST (14.4 ± 6.1 % TST). For stage N1 and N3 sleep, very low or very high values were reported for virtually all polysomnograms by different technologists. Technologists varied widely in their assignment of stage N3 sleep, scoring that stage when the digitally determined time of delta waves ranged from 3 to 17 seconds. Conclusions: Manual scoring of non-rapid eye movement sleep stages is highly unreliable among highly trained, experienced technologists. Measures of sleep continuity and depth that are reliable and clinically relevant should be a focus of clinical research.

Original languageEnglish (US)
Pages (from-to)205-213
Number of pages9
JournalJournal of Clinical Sleep Medicine
Volume14
Issue number2
DOIs
StatePublished - Feb 15 2018

Fingerprint

Sleep
Medicine
Sleep Stages
Eye Movements
Confidence Intervals
Research

Keywords

  • Digital sleep analysis
  • Interrater variability
  • Sleep depth

ASJC Scopus subject areas

  • Pulmonary and Respiratory Medicine
  • Neurology
  • Clinical Neurology

Cite this

Reliability of the American Academy of Sleep Medicine Rules for Assessing Sleep Depth in Clinical Practice. / Younes, Magdy; Kuna, Samuel T.; Pack, Allan I.; Walsh, James K.; Kushida, Clete A.; Staley, Bethany; Pien, Grace W.

In: Journal of Clinical Sleep Medicine, Vol. 14, No. 2, 15.02.2018, p. 205-213.

Research output: Contribution to journalArticle

Younes, Magdy ; Kuna, Samuel T. ; Pack, Allan I. ; Walsh, James K. ; Kushida, Clete A. ; Staley, Bethany ; Pien, Grace W. / Reliability of the American Academy of Sleep Medicine Rules for Assessing Sleep Depth in Clinical Practice. In: Journal of Clinical Sleep Medicine. 2018 ; Vol. 14, No. 2. pp. 205-213.
@article{14ef07fb96194158b5da7d183fb9a39e,
title = "Reliability of the American Academy of Sleep Medicine Rules for Assessing Sleep Depth in Clinical Practice",
abstract = "Study Objectives: The American Academy of Sleep Medicine has published manuals for scoring polysomnograms that recommend time spent in non-rapid eye movement sleep stages (stage N1, N2, and N3 sleep) be reported. Given the well-established large interrater variability in scoring stage N1 and N3 sleep, we determined the range of time in stage N1 and N3 sleep scored by a large number of technologists when compared to reasonably estimated true values. Methods: Polysomnograms of 70 females were scored by 10 highly trained sleep technologists, two each from five different academic sleep laboratories. Range and confidence interval (CI = difference between the 5th and 95th percentiles) of the 10 times spent in stage N1 and N3 sleep assigned in each polysomnogram were determined. Average values of times spent in stage N1 and N3 sleep generated by the 10 technologists in each polysomnogram were considered representative of the true values for the individual polysomnogram. Accuracy of different technologists in estimating delta wave duration was determined by comparing their scores to digitally determined durations. Results: The CI range of the ten N1 scores was 4 to 39 percent of total sleep time ({\%} TST) in different polysomnograms (mean CI ± standard deviation = 11.1 ± 7.1 {\%} TST). Corresponding range for N3 was 1 to 28 {\%} TST (14.4 ± 6.1 {\%} TST). For stage N1 and N3 sleep, very low or very high values were reported for virtually all polysomnograms by different technologists. Technologists varied widely in their assignment of stage N3 sleep, scoring that stage when the digitally determined time of delta waves ranged from 3 to 17 seconds. Conclusions: Manual scoring of non-rapid eye movement sleep stages is highly unreliable among highly trained, experienced technologists. Measures of sleep continuity and depth that are reliable and clinically relevant should be a focus of clinical research.",
keywords = "Digital sleep analysis, Interrater variability, Sleep depth",
author = "Magdy Younes and Kuna, {Samuel T.} and Pack, {Allan I.} and Walsh, {James K.} and Kushida, {Clete A.} and Bethany Staley and Pien, {Grace W.}",
year = "2018",
month = "2",
day = "15",
doi = "10.5664/jcsm.6934",
language = "English (US)",
volume = "14",
pages = "205--213",
journal = "Journal of Clinical Sleep Medicine",
issn = "1550-9389",
publisher = "American Academy of Sleep Medicine",
number = "2",

}

TY - JOUR

T1 - Reliability of the American Academy of Sleep Medicine Rules for Assessing Sleep Depth in Clinical Practice

AU - Younes, Magdy

AU - Kuna, Samuel T.

AU - Pack, Allan I.

AU - Walsh, James K.

AU - Kushida, Clete A.

AU - Staley, Bethany

AU - Pien, Grace W.

PY - 2018/2/15

Y1 - 2018/2/15

N2 - Study Objectives: The American Academy of Sleep Medicine has published manuals for scoring polysomnograms that recommend time spent in non-rapid eye movement sleep stages (stage N1, N2, and N3 sleep) be reported. Given the well-established large interrater variability in scoring stage N1 and N3 sleep, we determined the range of time in stage N1 and N3 sleep scored by a large number of technologists when compared to reasonably estimated true values. Methods: Polysomnograms of 70 females were scored by 10 highly trained sleep technologists, two each from five different academic sleep laboratories. Range and confidence interval (CI = difference between the 5th and 95th percentiles) of the 10 times spent in stage N1 and N3 sleep assigned in each polysomnogram were determined. Average values of times spent in stage N1 and N3 sleep generated by the 10 technologists in each polysomnogram were considered representative of the true values for the individual polysomnogram. Accuracy of different technologists in estimating delta wave duration was determined by comparing their scores to digitally determined durations. Results: The CI range of the ten N1 scores was 4 to 39 percent of total sleep time (% TST) in different polysomnograms (mean CI ± standard deviation = 11.1 ± 7.1 % TST). Corresponding range for N3 was 1 to 28 % TST (14.4 ± 6.1 % TST). For stage N1 and N3 sleep, very low or very high values were reported for virtually all polysomnograms by different technologists. Technologists varied widely in their assignment of stage N3 sleep, scoring that stage when the digitally determined time of delta waves ranged from 3 to 17 seconds. Conclusions: Manual scoring of non-rapid eye movement sleep stages is highly unreliable among highly trained, experienced technologists. Measures of sleep continuity and depth that are reliable and clinically relevant should be a focus of clinical research.

AB - Study Objectives: The American Academy of Sleep Medicine has published manuals for scoring polysomnograms that recommend time spent in non-rapid eye movement sleep stages (stage N1, N2, and N3 sleep) be reported. Given the well-established large interrater variability in scoring stage N1 and N3 sleep, we determined the range of time in stage N1 and N3 sleep scored by a large number of technologists when compared to reasonably estimated true values. Methods: Polysomnograms of 70 females were scored by 10 highly trained sleep technologists, two each from five different academic sleep laboratories. Range and confidence interval (CI = difference between the 5th and 95th percentiles) of the 10 times spent in stage N1 and N3 sleep assigned in each polysomnogram were determined. Average values of times spent in stage N1 and N3 sleep generated by the 10 technologists in each polysomnogram were considered representative of the true values for the individual polysomnogram. Accuracy of different technologists in estimating delta wave duration was determined by comparing their scores to digitally determined durations. Results: The CI range of the ten N1 scores was 4 to 39 percent of total sleep time (% TST) in different polysomnograms (mean CI ± standard deviation = 11.1 ± 7.1 % TST). Corresponding range for N3 was 1 to 28 % TST (14.4 ± 6.1 % TST). For stage N1 and N3 sleep, very low or very high values were reported for virtually all polysomnograms by different technologists. Technologists varied widely in their assignment of stage N3 sleep, scoring that stage when the digitally determined time of delta waves ranged from 3 to 17 seconds. Conclusions: Manual scoring of non-rapid eye movement sleep stages is highly unreliable among highly trained, experienced technologists. Measures of sleep continuity and depth that are reliable and clinically relevant should be a focus of clinical research.

KW - Digital sleep analysis

KW - Interrater variability

KW - Sleep depth

UR - http://www.scopus.com/inward/record.url?scp=85042076573&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85042076573&partnerID=8YFLogxK

U2 - 10.5664/jcsm.6934

DO - 10.5664/jcsm.6934

M3 - Article

C2 - 29351821

AN - SCOPUS:85042076573

VL - 14

SP - 205

EP - 213

JO - Journal of Clinical Sleep Medicine

JF - Journal of Clinical Sleep Medicine

SN - 1550-9389

IS - 2

ER -