Assessment of methodologic quality of randomized trials of interventional techniques: Development of an interventional pain management specific instrument

Laxmaiah Manchikanti, Joshua A. Hirsch, Steven Cohen, James E. Heavner, Frank J E Falco, Sudhir Diwan, Mark V. Boswell, Kenneth D. Candido, Obi Onyewu, Jie Zhu, Nalini Sehgal, Alan D. Kaye, Ramsin M. Benyamin, Standiford Helm, Vijay Singh, Sukdeb Datta, Sala Hadin Abdi, Paul Christo, Haroon Hameed, Mariam HameedRi Cardo Vallejo, Vidyasagar Pampati, Gabor B. Racz, P. Prithvi Raj

Research output: Contribution to journalArticle

Abstract

Background: A major component of a systematic review is an assessment of the methodological quality and bias of randomized trials. The most commonly utilized methodological quality assessment and bias assessment for randomized trials is by the Cochrane Review Group. While this is not a "gold standard," it is an indication of the current state-of-the-art review methodology. There is, however, no specific instrument to assess the methodological quality of manuscripts published for interventional techniques. Objectives: Our objective was to develop an instrument specifically for interventional pain management, to assess the methodological quality of randomized trials of interventional techniques. Methods: Item generation for the instrument was based on a definition of quality, to the extent to which the design and conduct of the trial were congruent with the objectives of the trial. Applicability was defined as the extent to which the trial produced procedures could be applied with contemporary interventional pain management techniques. Multiple items based on Cochrane review criteria were utilized along with specific requirements for interventional techniques. Results: A total of 22 items were developed which formed IPM-QRB or Interventional Pain Management Techniques - Quality Appraisal of Reliability and Risk of Bias Assessment tool. This included 9 of the 12 items from the Cochrane review criteria with definition of some items that were repetitive or duplicate, and the addition of 13 new items. The results were compared for inter-rater reliability of Cochrane review criteria and IPM-QRB, and inter- instrument reliability. The assessment was performed in multiple stages with an initial learning curve. The final assessment was for 4 randomized controlled trials (RCTs) utilizing both Cochrane review criteria and IPM-QRB criteria. The inter-rater agreement for Cochrane review criteria with overall intra-class correlation coefficient was 0.407 compared to an intra-class correlation coefficient of 0.833 for IPM-QRB criteria. The inter-rater agreement was superior for IPM-QRB criteria compared to Cochrane review criteria despite twice the items of Cochrane review criteria as IPM-QRB criteria with the detailed nature of assessment. Limitations: Limited validity or accuracy assessment of the instrument and the large number of items to be scored. Conclusion: We have developed a new comprehensive instrument to assess the methodological quality of randomized trials of interventional techniques. This instrument is superior to Cochrane review methodology criteria in that it provides more extensive and specific information for interventional techniques that will be useful in assessing the methodologic quality and bias of interventional techniques.

Original languageEnglish (US)
Pages (from-to)263-290
Number of pages28
JournalPain Physician
Volume17
Issue number3
StatePublished - 2014
Externally publishedYes

Fingerprint

Pain Management
Learning Curve
Manuscripts
Randomized Controlled Trials

Keywords

  • Cochrane reviews
  • Comparative effectiveness research
  • Evidence-based medicine
  • Interventional techniques
  • Methodological quality assessment
  • Risk of bias assessment

ASJC Scopus subject areas

  • Anesthesiology and Pain Medicine
  • Medicine(all)

Cite this

Manchikanti, L., Hirsch, J. A., Cohen, S., Heavner, J. E., Falco, F. J. E., Diwan, S., ... Prithvi Raj, P. (2014). Assessment of methodologic quality of randomized trials of interventional techniques: Development of an interventional pain management specific instrument. Pain Physician, 17(3), 263-290.

Assessment of methodologic quality of randomized trials of interventional techniques : Development of an interventional pain management specific instrument. / Manchikanti, Laxmaiah; Hirsch, Joshua A.; Cohen, Steven; Heavner, James E.; Falco, Frank J E; Diwan, Sudhir; Boswell, Mark V.; Candido, Kenneth D.; Onyewu, Obi; Zhu, Jie; Sehgal, Nalini; Kaye, Alan D.; Benyamin, Ramsin M.; Helm, Standiford; Singh, Vijay; Datta, Sukdeb; Abdi, Sala Hadin; Christo, Paul; Hameed, Haroon; Hameed, Mariam; Vallejo, Ri Cardo; Pampati, Vidyasagar; Racz, Gabor B.; Prithvi Raj, P.

In: Pain Physician, Vol. 17, No. 3, 2014, p. 263-290.

Research output: Contribution to journalArticle

Manchikanti, L, Hirsch, JA, Cohen, S, Heavner, JE, Falco, FJE, Diwan, S, Boswell, MV, Candido, KD, Onyewu, O, Zhu, J, Sehgal, N, Kaye, AD, Benyamin, RM, Helm, S, Singh, V, Datta, S, Abdi, SH, Christo, P, Hameed, H, Hameed, M, Vallejo, RC, Pampati, V, Racz, GB & Prithvi Raj, P 2014, 'Assessment of methodologic quality of randomized trials of interventional techniques: Development of an interventional pain management specific instrument', Pain Physician, vol. 17, no. 3, pp. 263-290.
Manchikanti, Laxmaiah ; Hirsch, Joshua A. ; Cohen, Steven ; Heavner, James E. ; Falco, Frank J E ; Diwan, Sudhir ; Boswell, Mark V. ; Candido, Kenneth D. ; Onyewu, Obi ; Zhu, Jie ; Sehgal, Nalini ; Kaye, Alan D. ; Benyamin, Ramsin M. ; Helm, Standiford ; Singh, Vijay ; Datta, Sukdeb ; Abdi, Sala Hadin ; Christo, Paul ; Hameed, Haroon ; Hameed, Mariam ; Vallejo, Ri Cardo ; Pampati, Vidyasagar ; Racz, Gabor B. ; Prithvi Raj, P. / Assessment of methodologic quality of randomized trials of interventional techniques : Development of an interventional pain management specific instrument. In: Pain Physician. 2014 ; Vol. 17, No. 3. pp. 263-290.
@article{81090bac551a4bb0947bfce333148611,
title = "Assessment of methodologic quality of randomized trials of interventional techniques: Development of an interventional pain management specific instrument",
abstract = "Background: A major component of a systematic review is an assessment of the methodological quality and bias of randomized trials. The most commonly utilized methodological quality assessment and bias assessment for randomized trials is by the Cochrane Review Group. While this is not a {"}gold standard,{"} it is an indication of the current state-of-the-art review methodology. There is, however, no specific instrument to assess the methodological quality of manuscripts published for interventional techniques. Objectives: Our objective was to develop an instrument specifically for interventional pain management, to assess the methodological quality of randomized trials of interventional techniques. Methods: Item generation for the instrument was based on a definition of quality, to the extent to which the design and conduct of the trial were congruent with the objectives of the trial. Applicability was defined as the extent to which the trial produced procedures could be applied with contemporary interventional pain management techniques. Multiple items based on Cochrane review criteria were utilized along with specific requirements for interventional techniques. Results: A total of 22 items were developed which formed IPM-QRB or Interventional Pain Management Techniques - Quality Appraisal of Reliability and Risk of Bias Assessment tool. This included 9 of the 12 items from the Cochrane review criteria with definition of some items that were repetitive or duplicate, and the addition of 13 new items. The results were compared for inter-rater reliability of Cochrane review criteria and IPM-QRB, and inter- instrument reliability. The assessment was performed in multiple stages with an initial learning curve. The final assessment was for 4 randomized controlled trials (RCTs) utilizing both Cochrane review criteria and IPM-QRB criteria. The inter-rater agreement for Cochrane review criteria with overall intra-class correlation coefficient was 0.407 compared to an intra-class correlation coefficient of 0.833 for IPM-QRB criteria. The inter-rater agreement was superior for IPM-QRB criteria compared to Cochrane review criteria despite twice the items of Cochrane review criteria as IPM-QRB criteria with the detailed nature of assessment. Limitations: Limited validity or accuracy assessment of the instrument and the large number of items to be scored. Conclusion: We have developed a new comprehensive instrument to assess the methodological quality of randomized trials of interventional techniques. This instrument is superior to Cochrane review methodology criteria in that it provides more extensive and specific information for interventional techniques that will be useful in assessing the methodologic quality and bias of interventional techniques.",
keywords = "Cochrane reviews, Comparative effectiveness research, Evidence-based medicine, Interventional techniques, Methodological quality assessment, Risk of bias assessment",
author = "Laxmaiah Manchikanti and Hirsch, {Joshua A.} and Steven Cohen and Heavner, {James E.} and Falco, {Frank J E} and Sudhir Diwan and Boswell, {Mark V.} and Candido, {Kenneth D.} and Obi Onyewu and Jie Zhu and Nalini Sehgal and Kaye, {Alan D.} and Benyamin, {Ramsin M.} and Standiford Helm and Vijay Singh and Sukdeb Datta and Abdi, {Sala Hadin} and Paul Christo and Haroon Hameed and Mariam Hameed and Vallejo, {Ri Cardo} and Vidyasagar Pampati and Racz, {Gabor B.} and {Prithvi Raj}, P.",
year = "2014",
language = "English (US)",
volume = "17",
pages = "263--290",
journal = "Pain Physician",
issn = "1533-3159",
publisher = "Association of Pain Management Anesthesiologists",
number = "3",

}

TY - JOUR

T1 - Assessment of methodologic quality of randomized trials of interventional techniques

T2 - Development of an interventional pain management specific instrument

AU - Manchikanti, Laxmaiah

AU - Hirsch, Joshua A.

AU - Cohen, Steven

AU - Heavner, James E.

AU - Falco, Frank J E

AU - Diwan, Sudhir

AU - Boswell, Mark V.

AU - Candido, Kenneth D.

AU - Onyewu, Obi

AU - Zhu, Jie

AU - Sehgal, Nalini

AU - Kaye, Alan D.

AU - Benyamin, Ramsin M.

AU - Helm, Standiford

AU - Singh, Vijay

AU - Datta, Sukdeb

AU - Abdi, Sala Hadin

AU - Christo, Paul

AU - Hameed, Haroon

AU - Hameed, Mariam

AU - Vallejo, Ri Cardo

AU - Pampati, Vidyasagar

AU - Racz, Gabor B.

AU - Prithvi Raj, P.

PY - 2014

Y1 - 2014

N2 - Background: A major component of a systematic review is an assessment of the methodological quality and bias of randomized trials. The most commonly utilized methodological quality assessment and bias assessment for randomized trials is by the Cochrane Review Group. While this is not a "gold standard," it is an indication of the current state-of-the-art review methodology. There is, however, no specific instrument to assess the methodological quality of manuscripts published for interventional techniques. Objectives: Our objective was to develop an instrument specifically for interventional pain management, to assess the methodological quality of randomized trials of interventional techniques. Methods: Item generation for the instrument was based on a definition of quality, to the extent to which the design and conduct of the trial were congruent with the objectives of the trial. Applicability was defined as the extent to which the trial produced procedures could be applied with contemporary interventional pain management techniques. Multiple items based on Cochrane review criteria were utilized along with specific requirements for interventional techniques. Results: A total of 22 items were developed which formed IPM-QRB or Interventional Pain Management Techniques - Quality Appraisal of Reliability and Risk of Bias Assessment tool. This included 9 of the 12 items from the Cochrane review criteria with definition of some items that were repetitive or duplicate, and the addition of 13 new items. The results were compared for inter-rater reliability of Cochrane review criteria and IPM-QRB, and inter- instrument reliability. The assessment was performed in multiple stages with an initial learning curve. The final assessment was for 4 randomized controlled trials (RCTs) utilizing both Cochrane review criteria and IPM-QRB criteria. The inter-rater agreement for Cochrane review criteria with overall intra-class correlation coefficient was 0.407 compared to an intra-class correlation coefficient of 0.833 for IPM-QRB criteria. The inter-rater agreement was superior for IPM-QRB criteria compared to Cochrane review criteria despite twice the items of Cochrane review criteria as IPM-QRB criteria with the detailed nature of assessment. Limitations: Limited validity or accuracy assessment of the instrument and the large number of items to be scored. Conclusion: We have developed a new comprehensive instrument to assess the methodological quality of randomized trials of interventional techniques. This instrument is superior to Cochrane review methodology criteria in that it provides more extensive and specific information for interventional techniques that will be useful in assessing the methodologic quality and bias of interventional techniques.

AB - Background: A major component of a systematic review is an assessment of the methodological quality and bias of randomized trials. The most commonly utilized methodological quality assessment and bias assessment for randomized trials is by the Cochrane Review Group. While this is not a "gold standard," it is an indication of the current state-of-the-art review methodology. There is, however, no specific instrument to assess the methodological quality of manuscripts published for interventional techniques. Objectives: Our objective was to develop an instrument specifically for interventional pain management, to assess the methodological quality of randomized trials of interventional techniques. Methods: Item generation for the instrument was based on a definition of quality, to the extent to which the design and conduct of the trial were congruent with the objectives of the trial. Applicability was defined as the extent to which the trial produced procedures could be applied with contemporary interventional pain management techniques. Multiple items based on Cochrane review criteria were utilized along with specific requirements for interventional techniques. Results: A total of 22 items were developed which formed IPM-QRB or Interventional Pain Management Techniques - Quality Appraisal of Reliability and Risk of Bias Assessment tool. This included 9 of the 12 items from the Cochrane review criteria with definition of some items that were repetitive or duplicate, and the addition of 13 new items. The results were compared for inter-rater reliability of Cochrane review criteria and IPM-QRB, and inter- instrument reliability. The assessment was performed in multiple stages with an initial learning curve. The final assessment was for 4 randomized controlled trials (RCTs) utilizing both Cochrane review criteria and IPM-QRB criteria. The inter-rater agreement for Cochrane review criteria with overall intra-class correlation coefficient was 0.407 compared to an intra-class correlation coefficient of 0.833 for IPM-QRB criteria. The inter-rater agreement was superior for IPM-QRB criteria compared to Cochrane review criteria despite twice the items of Cochrane review criteria as IPM-QRB criteria with the detailed nature of assessment. Limitations: Limited validity or accuracy assessment of the instrument and the large number of items to be scored. Conclusion: We have developed a new comprehensive instrument to assess the methodological quality of randomized trials of interventional techniques. This instrument is superior to Cochrane review methodology criteria in that it provides more extensive and specific information for interventional techniques that will be useful in assessing the methodologic quality and bias of interventional techniques.

KW - Cochrane reviews

KW - Comparative effectiveness research

KW - Evidence-based medicine

KW - Interventional techniques

KW - Methodological quality assessment

KW - Risk of bias assessment

UR - http://www.scopus.com/inward/record.url?scp=84901258145&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84901258145&partnerID=8YFLogxK

M3 - Article

C2 - 24850111

AN - SCOPUS:84901258145

VL - 17

SP - 263

EP - 290

JO - Pain Physician

JF - Pain Physician

SN - 1533-3159

IS - 3

ER -