Five AI systems (EAU Guidelines Bot, ChatGPT-5, Gemini 2.5 Pro, Copilot - Smart GPT-5, and Perplexity Pro) were evaluated on 13 clinical questions derived from strongly recommended EAU erectile dysfunction (ED) guideline statements, scored by three senior reviewers across five domains (relevance, clarity, structure, clinical utility, factual accuracy) on a 5-point Likert scale.
Gemini 2.5 Pro and EAU Guidelines Bot achieved the highest composite scores [4.60 and 4.53, respectively], followed by ChatGPT-5 [4.27]; Copilot - Smart GPT-5 [3.73] and Perplexity Pro [3.60] scored lower; all inter-model differences were statistically significant (p < 0.001), with factual accuracy consistently high (≥4) among top performers but notable variability in clarity, structure, and clinical utility.
- Only 13 questions were used, all drawn from 'strongly recommended' EAU statements, limiting generalizability to nuanced or weakly recommended guideline areas. - Reviewer scoring was qualitative (Likert scale) without real-world patient outcome validation. - No external validation in actual clinical or patient-facing settings was performed.
Clinicians exploring AI tools for ED guideline support may find Gemini 2.5 Pro and the EAU Guidelines Bot perform best in structured scenarios, but all models should be used as adjuncts—not replacements—for clinical judgment, given variability in structure and clinical utility scores.