Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists

Tatyana A. Shamliyan; Robert L. Kane; Mohammed T. Ansari; Gowri Raman; Nancy D. Berkman; Mark Grant; Gail Janes; Margaret Maglione; David Moher; Mona Nasser; Karen A. Robinson; Jodi B. Segal; Sophia Tsouros

doi:10.1016/j.jclinepi.2010.08.006

Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists

Tatyana A. Shamliyan, Robert L. Kane, Mohammed T. Ansari, Gowri Raman, Nancy D. Berkman, Mark Grant, Gail Janes, Margaret Maglione, David Moher, Mona Nasser, Karen A. Robinson, Jodi B. Segal, Sophia Tsouros

School of Medicine

Research output: Contribution to journal › Article › peer-review

76 Scopus citations

Abstract

Objective: To develop two checklists for the quality of observational studies of incidence or risk factors of diseases. Study Design and Setting: Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles. Results: The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers. Conclusion: We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.

Original language	English (US)
Pages (from-to)	637-657
Number of pages	21
Journal	Journal of Clinical Epidemiology
Volume	64
Issue number	6
DOIs	https://doi.org/10.1016/j.jclinepi.2010.08.006
State	Published - Jun 2011

Keywords

Bias (epidemiology)
Morbidity
Quality control
Reproducibility of results
Review literature as topic
Risk factors
Validation studies

ASJC Scopus subject areas

Epidemiology

Access to Document

10.1016/j.jclinepi.2010.08.006

Cite this

Shamliyan, T. A., Kane, R. L., Ansari, M. T., Raman, G., Berkman, N. D., Grant, M., Janes, G., Maglione, M., Moher, D., Nasser, M., Robinson, K. A., Segal, J. B., & Tsouros, S. (2011). Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists. Journal of Clinical Epidemiology, 64(6), 637-657. https://doi.org/10.1016/j.jclinepi.2010.08.006

Shamliyan, TA, Kane, RL, Ansari, MT, Raman, G, Berkman, ND, Grant, M, Janes, G, Maglione, M, Moher, D, Nasser, M, Robinson, KA , Segal, JB & Tsouros, S 2011, 'Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists', Journal of Clinical Epidemiology, vol. 64, no. 6, pp. 637-657. https://doi.org/10.1016/j.jclinepi.2010.08.006

@article{7dc096f3d8b74f38a60434dc3c7277a8,

title = "Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists",

abstract = "Objective: To develop two checklists for the quality of observational studies of incidence or risk factors of diseases. Study Design and Setting: Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles. Results: The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers. Conclusion: We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.",

keywords = "Bias (epidemiology), Morbidity, Quality control, Reproducibility of results, Review literature as topic, Risk factors, Validation studies",

author = "Shamliyan, {Tatyana A.} and Kane, {Robert L.} and Ansari, {Mohammed T.} and Gowri Raman and Berkman, {Nancy D.} and Mark Grant and Gail Janes and Margaret Maglione and David Moher and Mona Nasser and Robinson, {Karen A.} and Segal, {Jodi B.} and Sophia Tsouros",

note = "Funding Information: Bush DE, Ziegelstein RC, Patel UV, Thombs BD, Ford DE, Fauerbach JA, et al. Post-Myocardial Infarction Depression. Evidence Report/Technology Assessment No. 123. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 290-02-0018.) AHRQ Publication No. 05-E018-2. Rockville, MD: Agency for Healthcare Research and Quality. May 2005. Funding Information: Buscemi N, Vandermeer B, Friesen C, Bialy L, Tubman M, Ospina M, et al. Manifestations and Management of Chronic Insomnia in Adults. Evidence Report/Technology Assessment No. 125. (Prepared by the University of Alberta Evidence-based Practice Center, under Contract No. C400000021.) AHRQ Publication No. 05-E021-2. Rockville, MD: Agency for Healthcare Research and Quality. June 2005. ",

year = "2011",

month = jun,

doi = "10.1016/j.jclinepi.2010.08.006",

language = "English (US)",

volume = "64",

pages = "637--657",

journal = "Journal of Clinical Epidemiology",

issn = "0895-4356",

publisher = "Elsevier USA",

number = "6",

}

TY - JOUR

T1 - Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases

T2 - Pilot study of new checklists

AU - Shamliyan, Tatyana A.

AU - Kane, Robert L.

AU - Ansari, Mohammed T.

AU - Raman, Gowri

AU - Berkman, Nancy D.

AU - Grant, Mark

AU - Janes, Gail

AU - Maglione, Margaret

AU - Moher, David

AU - Nasser, Mona

AU - Robinson, Karen A.

AU - Segal, Jodi B.

AU - Tsouros, Sophia

N1 - Funding Information: Bush DE, Ziegelstein RC, Patel UV, Thombs BD, Ford DE, Fauerbach JA, et al. Post-Myocardial Infarction Depression. Evidence Report/Technology Assessment No. 123. (Prepared by the Johns Hopkins University Evidence-based Practice Center under Contract No. 290-02-0018.) AHRQ Publication No. 05-E018-2. Rockville, MD: Agency for Healthcare Research and Quality. May 2005. Funding Information: Buscemi N, Vandermeer B, Friesen C, Bialy L, Tubman M, Ospina M, et al. Manifestations and Management of Chronic Insomnia in Adults. Evidence Report/Technology Assessment No. 125. (Prepared by the University of Alberta Evidence-based Practice Center, under Contract No. C400000021.) AHRQ Publication No. 05-E021-2. Rockville, MD: Agency for Healthcare Research and Quality. June 2005.

PY - 2011/6

Y1 - 2011/6

N2 - Objective: To develop two checklists for the quality of observational studies of incidence or risk factors of diseases. Study Design and Setting: Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles. Results: The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers. Conclusion: We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.

AB - Objective: To develop two checklists for the quality of observational studies of incidence or risk factors of diseases. Study Design and Setting: Initial development of the checklists was based on a systematic literature review. The checklists were refined after pilot trials of validity and reliability were conducted by seven experts, who tested the checklists on 10 articles. Results: The checklist for studies of incidence or prevalence of chronic disease had six criteria for external validity and five for internal validity. The checklist for risk factor studies had six criteria for external validity, 13 criteria for internal validity, and two aspects of causality. A Microsoft Access database produced automated standardized reports about external and internal validities. Pilot testing demonstrated face and content validities and discrimination of reporting vs. methodological qualities. Interrater agreement was poor. The experts suggested future reliability testing of the checklists in systematic reviews with preplanned protocols, a priori consensus about research-specific quality criteria, and training of the reviewers. Conclusion: We propose transparent and standardized quality assessment criteria of observational studies using the developed checklists. Future testing of the checklists in systematic reviews is necessary to develop reliable tools that can be used with confidence.

KW - Bias (epidemiology)

KW - Morbidity

KW - Quality control

KW - Reproducibility of results

KW - Review literature as topic

KW - Risk factors

KW - Validation studies

UR - http://www.scopus.com/inward/record.url?scp=79955019836&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79955019836&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2010.08.006

DO - 10.1016/j.jclinepi.2010.08.006

M3 - Article

C2 - 21071174

AN - SCOPUS:79955019836

SN - 0895-4356

VL - 64

SP - 637

EP - 657

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

IS - 6

ER -

Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: Pilot study of new checklists

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this