Module 1 – Lesson 5 – Appendix B

Sample Critical Appraisal of a Clinical Practice Guideline

Critical Appraisal: Clinical Practice Guideline 

Note: This sample is intended to highlight the most important features of this type of critical appraisal. It is not all-inclusive and does not represent or reflect any specific currently-available critical appraisal tool.

Title:

The Treatment of Restless Legs Syndrome (RLS) and Periodic Limb Movement Disorder (PLMD) in Adults: Update for 2012:  Practice parameters with an Evidence-based Systematic Review and Meta-Analysis

Authors:

  • Aurora, R. N., Kristo, D. A., Bista, S. R., Rowley, J. A., Zak, R. S., Casey, K. R., Lam, C., Tracy, S. and Rosenberg, R. S.  (2012). (American Academy of Sleep Medicine CPG Group)

Reference:

  • Sleep, 35(8), 1039-1062
QUESTION RESPONSE COMMENTS
Is the group, committee or organizationthat developed the guidelinesidentified? Yes Task force membersof theAmerican Academy of Sleep Medicine were identified on Page 1 with their detailed affiliations.
-Have the guidelines beensponsored or funded by an external group with potential conflict of interest?
-Have the guidelines beenendorsed or supported by an external organization?		 Yes Page1055 (just before references); disclosure statement re: not an industry supported study and no financial conflicts of interest. Critique was undertaken by an external source. Endorsed by American Academy of Sleep Medicine).
Does the document indicateWHY they undertook this guidelinedevelopment work, or is it a revision of previous work? Yes Whiletwo previous (limited) works had been conducted on these rare disease entities, authors noted that an updateof 2003 and 2004 was needed to support consistency and to include literature (page 1) and aneed was expressed for consistency of screening tools (page 2).
Is theoverall Objective, Purpose and Question (PICOs)for guideline stated? Yes Yes, Table 1, page 1041 outlines the multiple PICO questions undertaken/formulated and defined clearly that guidelines foradults only, and specific drug classes vs. control.
-Is asystematic literature search described? -Is there mention of how relevantarticles were selected or excluded for review? Yes Yes, processes for literature search were described on page 1040 with MeSH terms and limitations defined. Exclusions included studies with <5 subjects and treatments <1 week. English only adults only, randomized controlled trials only, no editorials, letters, or case reports (bottom of 1040). Of note: this document is both the comprehensive systematic review and also a CPG; the group opted to create the same thing all at once. They could do so because their clinical topic area was very finite/limited, and it was a somewhat less-common condition. Other topics such as cardiovascularor diabetic topics could never be effectively handled this way.
Did authors declareconflicts of interestamong all parties involved in guideline preparation and consensus? Yes Last page: declaration of no conflicts of interest and no industry funding.
Is it clear howconsensus for evidence rating and recommendation preparationwas reached? Yes GRADE process was used as a process for rating quality of evidence; Page 1041: all studies were assessed by two task force members for design and limitations to validate bias; Box 1 reflects the final assessments of evidence. Tables 1-3 outline processes for same.
Havepatient viewpoints(or those who will bemost affected by these guidelines) been sought? No This is not specified and anticipated to be “no” although one “flag” that should always be kept in mind with less-common clinical topics or conditions is that there’s always that risk for expert viewpoints to find their way into guidelines anyway, even when steps have been taken to mitigate this). The reason is that there just isn’t ample research available on all entities of care for less-common conditions (such as restless legs syndrome, as one example), so there will always be aspects were viewpoints of experts will just filter in.
Wereevidence ratings or indicators of value indicated for each recommendationor guidance statement? Yes Table 4 captures in simplified language the key results with strength of recommendation and body of evidence level noted for each practice parameter.
Are therehints of bias or untoward influencein the process? No There does not appear to be. Therapies are summarized based on ‘classes’ presentation and included pharmacologic and non-pharmacologic options (CBT, exercise and placebo effect)
Is theoverall Objective, Purpose and Question (PICOTs)for guideline stated? Yes Yes, Table 1, page 1041 outlines the multiple PICO questions undertaken/formulated and defined clearly that guidelines for ADULTS only, and specific drug classes vs control.
-Is asystematic literature search described? -Is there mention of how relevantarticles were selected or excluded for review? Yes Yes, processes for literature search were described on page 1040 with MeSH terms and limitations defined. Exclusions included studies with <5 subjects and treatments <1 week. English only adults only, RCTs only, no editorials, letters or case reports (bottom of 1040)
Of note….this document is both the comprehensive systematic review (SR) and also a CPG…the group opted to “kill 2 birds with 1 stone” in creating same all at once. They could do so because their clinical topic area was very finite/limited and it was a somewhat less-common condition. Other topics such as CV or diabetic topics could never be effectively handled this way.
Did authors declareconflicts of interestamong all parties involved in guideline preparation and consensus? Yes Last page declaration of no conflicts of interest and no industry funding.
Is it clear howconsensus for evidence rating and recommendation preparationwas reached? Yes GRADE process was used as a process for rating quality of evidence; Page 1041: all studies were assessed by 2 task force members for design and limitations to validate bias; Box 1 reflects the final assessments of evidence. Tables 1-3 outline processes for same.
Havepatient viewpoints(or those who will bemost impacted by these guidelines) been sought? No This is not specified and anticipated to be “no” although one “flag” that we should always keep in mind with less-common clinical topics or conditions is that there’s always that risk for expert viewpoints to find their way into guidelines anyway, even when steps have been taken to mitigate this). Reason is that there just isn’t ample research available on all entities of care for less-common conditions (such as restless leg, as one example), so there will always be aspects were viewpoints of experts will just filter in.
Wereevidence ratings or indicators of value indicated for each recommendationor guidance statement? Yes Table 4 captures in simplified language the key results with strength of recommendation and body of evidence level noted for each practice parameter.
Are therehints of bias or untoward influencein the process? No There does not appear to be. Therapies are summarized based on classes presentation and included pharmacologic and non-pharmacologic options (cognitive behavioural therapy, exercise and placebo effect).
What are the results?
QUESTION RESPONSE COMMENTS
Are themessages from guidelines clear and easily understandable to the end-user (policy-maker, clinician, patient)? Yes Table 4 presents all recommendations as a 1-page reference guide for clinicians. Harm/burden assessment ratings are included and FDA status on drug classes arediscussed.
Are there direct andeasy to follow linkages between guideline ratingsand raw data/evidenceused to create them? Yes Yes, as above.
Is there discussion ofbenefits, harm, risks and cost impacts? Yes Yes, harms and benefits included in Table 4 and are highlighted along with individual study results in forest plots for individual drug classes/therapies. There isno discussion of cost impact, however.
In sections wherebyexpert consensus was requiredorno evidence was found, are processes to arrive at ratings outlined? Can’t tell This is not clearly defined and expected/assumed to be through committee discussions. There was mention of several areas where no evidence was found, thus no CPG was prepared in most cases.
Will the results help me in my decision-making?
QUESTION RESPONSE COMMENTS
Are the guidelines presented in auser-friendly and easy to follow format?(Are they ACTIONABLE?) Yes Table 4 supports a user-friendly format for clinicians to use as areference. Not every CPG has this spelled out in this simplistic way. It is because the topic is limited/finite and not interconnected with other conditions that this can occur with relative ease in this guideline. One should not expect all guidelines to be this simple or straightforward.
Do the guidelines offernext-steps for practical implementation; is there recognition of implementation barriers? No This is not completely clear within the recommendations, and there are no suggestions for knowledge transfer or implementation strategies.
Is thereadvice presented on how to audit for compliance or qualityimprovement impact? No Not noted in this document.
Is there supportingpatient and family (plain language) versions or education materialsavailable? No Does not appear to be included.
Is there discussion ofclinical flexibility for application in multiple or diverse clinical settings? Yes Options were presented based on evidence presented and summarized within Table 4
Is there atimeline or plan for updatesto these guidelines? Can’t tell Not stipulated. The authors described the reason forthis update wasdue to the large amount of literature on dopaminergic medications for RLS and newer therapies.