ReachRxResearch
Home
Research
Sign Up
Quality assessment of artificial intelligence responses in erectile dysfunction: a comparative study based on EAU recommendations
Back to Research Hub
  1. Home
  2. Research Hub
  3. Quality assessment of artificial intelli…

Quality assessment of artificial intelligence responses in erectile dysfunction: a comparative study based on EAU recommendations

International Journal of Impotence Research·August 7
Urology & NephrologyLimited evidenceErectile DysfunctionComparative StudyArtificial Intelligence / Large Language ModelAdult

Summary

View source

What was studied

Five AI systems (EAU Guidelines Bot, ChatGPT-5, Gemini 2.5 Pro, Copilot - Smart GPT-5, and Perplexity Pro) were evaluated on 13 clinical questions derived from strongly recommended EAU erectile dysfunction (ED) guideline statements, scored by three senior reviewers across five domains (relevance, clarity, structure, clinical utility, factual accuracy) on a 5-point Likert scale.

Key findings

Gemini 2.5 Pro and EAU Guidelines Bot achieved the highest composite scores [4.60 and 4.53, respectively], followed by ChatGPT-5 [4.27]; Copilot - Smart GPT-5 [3.73] and Perplexity Pro [3.60] scored lower; all inter-model differences were statistically significant (p < 0.001), with factual accuracy consistently high (≥4) among top performers but notable variability in clarity, structure, and clinical utility.

Study limitations

- Only 13 questions were used, all drawn from 'strongly recommended' EAU statements, limiting generalizability to nuanced or weakly recommended guideline areas. - Reviewer scoring was qualitative (Likert scale) without real-world patient outcome validation. - No external validation in actual clinical or patient-facing settings was performed.

Clinical implications

Clinicians exploring AI tools for ED guideline support may find Gemini 2.5 Pro and the EAU Guidelines Bot perform best in structured scenarios, but all models should be used as adjuncts—not replacements—for clinical judgment, given variability in structure and clinical utility scores.

Caveats

  • Sample size tag reflects the number of AI systems compared (n=5), not a patient population — the study has no human subjects enrolled.
  • Scores reflect structured scenario performance only; real-world clinical utility of these AI tools remains unvalidated.
  • Study design is classified as a structured comparative/benchmarking study; it does not fit neatly into standard clinical trial hierarchies, so level of evidence is approximate.

Related Questions

Explore related topics

How do AI chatbots compare to clinical guidelines in urology decision support?What are the best AI tools for answering erectile dysfunction clinical questions?How reliable are large language models for interpreting EAU urology guidelines?

Publication Details

Year
2026
Journal
International Journal of Impotence Research
Sample Size
n=5
Source
View article
Keep scrolling
More content below.
Up Next