A randomised trial comparing three Delphi feedback strategies found no evidence of a difference in a setting with high initial agreement

Objective: To explore the impact of different feedback strategies on 1) subsequent agreement and 2) variability in Delphi studies.
Study Design and Setting: A two-round Delphi survey, with a list of outcomes generated from the results of a systematic review and interviews, was undertaken whilst developing a core outcomes set for prostate cancer including two stakeholder groups (health professionals and patients). Seventy-nine outcomes were scored on a scale of one (not important) to nine (critically important). Participants were randomised in round two to receive round one feedback from: peers only, multiple stakeholders separately, or multiple stakeholders combined.
Results: Agreement on outcomes retained for all feedback groups was high (peer: 92%, multiple separate: 90%, multiple combined: 84%). There were no statistically significant reduction in variability for peer versus multiple-separate (0.016 (-0.035, 0.067); p=0.529), or multiple-separate versus multiple-combined feedback (0.063 (-0.003, 0.129); p=0.062). Peer feedback statistically significantly reduced variability compared to multiple-combined feedback (0.079 (0.001, 0.157); p=0.046).
Conclusions: We found no evidence of a difference between different feedback strategies in terms of the number of outcomes retained or reduction in variability of opinion. However, this may be explained by the high level of existing agreement in round one. Further methodological studies nested within Delphi surveys will help clarify the best strategy.