Assessment of methodologic quality of randomized trials of interventional techniques: Development of an interventional pain management specific instrument

Laxmaiah Manchikanti, Joshua A. Hirsch, Steven P. Cohen, James E. Heavner, Frank J.E. Falco, Sudhir Diwan, Mark V. Boswell, Kenneth D. Candido, Obi Onyewu, Jie Zhu, Nalini Sehgal, Alan D. Kaye, Ramsin M. Benyamin, Standiford Helm, Vijay Singh, Sukdeb Datta, Sala Hadin Abdi, Paul J. Christo, Haroon Hameed, Mariam HameedRi Cardo Vallejo, Vidyasagar Pampati, Gabor B. Racz, P. Prithvi Raj

Research output: Contribution to journalArticlepeer-review

Abstract

Background: A major component of a systematic review is an assessment of the methodological quality and bias of randomized trials. The most commonly utilized methodological quality assessment and bias assessment for randomized trials is by the Cochrane Review Group. While this is not a "gold standard," it is an indication of the current state-of-the-art review methodology. There is, however, no specific instrument to assess the methodological quality of manuscripts published for interventional techniques. Objectives: Our objective was to develop an instrument specifically for interventional pain management, to assess the methodological quality of randomized trials of interventional techniques. Methods: Item generation for the instrument was based on a definition of quality, to the extent to which the design and conduct of the trial were congruent with the objectives of the trial. Applicability was defined as the extent to which the trial produced procedures could be applied with contemporary interventional pain management techniques. Multiple items based on Cochrane review criteria were utilized along with specific requirements for interventional techniques. Results: A total of 22 items were developed which formed IPM-QRB or Interventional Pain Management Techniques - Quality Appraisal of Reliability and Risk of Bias Assessment tool. This included 9 of the 12 items from the Cochrane review criteria with definition of some items that were repetitive or duplicate, and the addition of 13 new items. The results were compared for inter-rater reliability of Cochrane review criteria and IPM-QRB, and inter- instrument reliability. The assessment was performed in multiple stages with an initial learning curve. The final assessment was for 4 randomized controlled trials (RCTs) utilizing both Cochrane review criteria and IPM-QRB criteria. The inter-rater agreement for Cochrane review criteria with overall intra-class correlation coefficient was 0.407 compared to an intra-class correlation coefficient of 0.833 for IPM-QRB criteria. The inter-rater agreement was superior for IPM-QRB criteria compared to Cochrane review criteria despite twice the items of Cochrane review criteria as IPM-QRB criteria with the detailed nature of assessment. Limitations: Limited validity or accuracy assessment of the instrument and the large number of items to be scored. Conclusion: We have developed a new comprehensive instrument to assess the methodological quality of randomized trials of interventional techniques. This instrument is superior to Cochrane review methodology criteria in that it provides more extensive and specific information for interventional techniques that will be useful in assessing the methodologic quality and bias of interventional techniques.

Original languageEnglish (US)
Pages (from-to)263-290
Number of pages28
JournalPain physician
Volume17
Issue number3
StatePublished - 2014
Externally publishedYes

Keywords

  • Cochrane reviews
  • Comparative effectiveness research
  • Evidence-based medicine
  • Interventional techniques
  • Methodological quality assessment
  • Risk of bias assessment

ASJC Scopus subject areas

  • Anesthesiology and Pain Medicine

Fingerprint Dive into the research topics of 'Assessment of methodologic quality of randomized trials of interventional techniques: Development of an interventional pain management specific instrument'. Together they form a unique fingerprint.

Cite this