AGREE II Quality assessment of National and International Clinical Practice Guidelines on Prostate Cancer Management by the OPTIMA Consortium

Background and Objective: Clinical practice guidelines Clinical Practice Guidelines for prostate cancer (PCa) are a valuable resource for everyday clinical practice. The Clinical Practice Guidelines and recommendations produced by various societies should demonstrate a considerable level of consistency in terms of quality, regardless of the society that developed them given the common evidence base. However, to date, no study has assessed the quality of PCa Clinical Practice Guidelines. As part of the OPTIMA project, we evaluated the quality of the most frequently used national and international Clinical Practice Guidelines for PCa using the Appraisal of Guidelines for Research & Evaluation II (AGREE II) tool.
Method: The quality of the identified Clinical Practice Guidelines was assessed independently by two assessors using the AGREE II tool. The AGREE II comprises 23 different items organized into 6 domains, rated
on a 7-point scale (1: strongly disagree to 7: strongly agree).The total score of the appraisal was the mean value of the two assessments. The agreement between assessors’ scores was calculated using the Interclass correlation coefficient. Four key recommendations were compared among the included Clinical Practice
Guidelines to assess consistency.
Key Findings and Limitations: Sixteen Clinical Practice Guidelines were assessed using their latest available version (cut off April 2024). The EAU, S3LL PCa, KCE, NCCN, and NPV guidelines received the highest overall scores with mean domain score 82.4% (range:75.5-88.3%). The AFU, AUA, and NICE received a mean domain score 77.6% (range:73.7-84.0%). Below average were the ESMO, the localised and systemic CPPC ASCO, the NVU with mean domain score 58.4% (range:43.5-76.3%). Reasons for scoring below average included: inadequate information about the methodology applied, limited scope of the guideline and
limited patient engagement. The highest interrater variability was observed in NVU(ICC:0.58) and the lowest in AFU-L (ICC:0.84). When examining the scores of each domain, ‘Clarity of Presentation’ (Domain 4) achieved the highest score with a mean of 86.9% ±12.6. The domain with the lowest score was Applicability
(Domain 5), with a mean of 48.3% ± 24.8. The ICC was calculated 0.72 (±0.08).
Conclusion: This is the first study in which comprehensive quality assessment of majority International and National Clinical Practice Guidelines was undertaken, and key recommendations were compared to assess consistency. Our study shows that the majority of International and National Clinical Practice Guidelines demonstrate high-quality standards when assessed using the AGREE II evaluation tool. Those Clinical Practice Guidelines that did not meet the expected standards could be improved by adopting several key
recommendations outlined by our study.